Why assigning custom data view IDs matters in Kibana

I was recently introduced to a TV show called Young Sheldon. I’ve never seen The Big Bang Theory, so I wasn’t sure what to expect. That said, this post isn’t really about the show. I’ll be using a dataset related to it, but the actual data could’ve been anything: business-related, observability-related, security-related — you name it. For those who know me, you know I’m always going to find a way to infuse a little semi-chaotic fun. So, let’s get into it.

https://static-www.elastic.co/v3/assets/bltefdd0b53724fa2ce/bltbb9f6702b3952fc5/6841a4da116bcfc2812211a1/blog-create-data-view.png,blog-create-data-view.png

Sheldon selects the data view and confirms that the custom data view ID is correctly set.

Note: This can be quickly verified through the browser URL.

https://static-www.elastic.co/v3/assets/bltefdd0b53724fa2ce/blt53cf11c336fb81a8/684095ad954d3bd311942d2e/demo-young-sheldon.png,demo-young-sheldon.png

Georgie creates a data view using the demo-young-sheldon index. The following parameters are used:

  • Name: demo-young-sheldon-test

  • Index Pattern: demo-young-sheldon

  • Custom Data View ID: N/A (Blank)
https://static-www.elastic.co/v3/assets/bltefdd0b53724fa2ce/blt669881eafdfb0aff/684096e9ba5842bf4b762942/demo-young-sheldon-test-index.png,demo-young-sheldon-test-index.png

Georgie selects the data view and confirms that the data view ID, because it was left blank, has been set to the random automatically generated string.

Note: This can be quickly verified through the browser URL.

https://static-www.elastic.co/v3/assets/bltefdd0b53724fa2ce/blt90e9e6e5c9713c8e/68409741bac1ae46593cf25f/demo-young-sheldon-test.png,demo-young-sheldon-test.png

Sheldon (demo-young-sheldon) and Georgie (demo-young-sheldon-test) each create visualizations using their respective data views and add them to a shared dashboard.

https://static-www.elastic.co/v3/assets/bltefdd0b53724fa2ce/bltdc52a2f50331187d/6840981bb705e01d836423f0/demo-young-sheldon-two-dashboards.png,demo-young-sheldon-two-dashboards.png

Sheldon and Georgie accidentally delete their data views and when they navigate back to the dashboard, they notice their visualizations are broken.

https://static-www.elastic.co/v3/assets/bltefdd0b53724fa2ce/blt0a8a08c439fc0d7a/68409895a38f5d6358ec7a19/could-not-find-data-view.png,could-not-find-data-view.png

Sheldon strategically assigned a custom data view ID to his data view, making recovery simple. To restore his visualization, he simply recreates the data view and reuses the same custom data view ID: demo-young-sheldon.

The following parameters are used:.

  • Name: demo-young-sheldon-recreated

  • Index Pattern: demo-young-sheldon

  • Custom Data View ID: demo-young-sheldon

Note: In this walkthrough, I’ve slightly modified the data view name for clarity and to differentiate it from earlier steps (Name: demo-young-sheldon → demo-young-sheldon-recreated).

https://static-www.elastic.co/v3/assets/bltefdd0b53724fa2ce/blt6f610978462863d9/68409902f809c156803e3636/create-data-view.png,create-data-view.png

Sheldon returns to the shared dashboard and confirms that his visualization has been successfully restored.

Georgie can follow a similar process to restore his visualization. However, he must set the custom data view ID to match the randomly generated string shown in the image on the right.

https://static-www.elastic.co/v3/assets/bltefdd0b53724fa2ce/bltce1854169b0b0a42/684099a6d2161d4e8ad3ac20/dashboard-demo-young-sheldon.png,dashboard-demo-young-sheldon.pngYou're thinking it and you're not entirely wrong (but not right either)

You might be thinking, "Well, I could just recreate the `demo-young-sheldon-test data view and manually assign the same randomly generated data view ID referenced in the visualization."

https://static-www.elastic.co/v3/assets/bltefdd0b53724fa2ce/blt9f54b7d2477e1361/68409a0264e286f168bc6b94/hmmm.png,hmmm.png

And you’d be absolutely right. That approach does work — in theory.

But now, let’s scale the scenario.

Imagine you have 30 users, and 10 of them each create their own variation of the demo-young-sheldon data view, each with a different, randomly generated data view ID. They build hundreds of visualizations using these various data views, which are then distributed across multiple dashboards.

Later, someone reviewing the environment sees 10 data views with similar names and the same index pattern associated with all of them. Assuming they’re duplicates, they delete 9 of them unaware that each one is uniquely tied to specific visualizations via data view ID.

Now, you’ve got a problem. Many of these visualizations are spread across multiple dashboards, and in many cases, the users are unaware that the visualizations within a single dashboard are not using the same data view. For example, a user notices the following data views: zeek-1, zeek-2, zeek-3, zeek-4, and zeek-5.

https://static-www.elastic.co/v3/assets/bltefdd0b53724fa2ce/blt2308920da1dc30aa/68409a63bac1aeeccc3cf288/Data-views.png,Data-views.png

Since all of the data views point to the same index pattern, users often don’t think twice when building a visualization and adding it to a shared dashboard.

https://static-www.elastic.co/v3/assets/bltefdd0b53724fa2ce/blt5e3f4bf3429d70cb/68409a8a7f9f7313e67dfe4c/count-of-records-number.png,count-of-records-number.pnghttps://static-www.elastic.co/v3/assets/bltefdd0b53724fa2ce/blt72c2bed3e2d0b149/68409ad73010e32dc0e23193/data-view-zeek-2.png,data-view-zeek-2.pnghttps://static-www.elastic.co/v3/assets/bltefdd0b53724fa2ce/bltd78a40a0de2c00a9/68409adf98bc4e45fa8190e6/Data-view-zeek-3.png,Data-view-zeek-3.pnghttps://static-www.elastic.co/v3/assets/bltefdd0b53724fa2ce/blta94dcc27b15dbcb9/68409af198bc4e1e708190ed/data-view-zeek-4.png,data-view-zeek-4.pnghttps://static-www.elastic.co/v3/assets/bltefdd0b53724fa2ce/blt4c298021d979e05a/68409b047f9f730a8b7dfe54/data-view-zeek-5.png,data-view-zeek-5.png

As a result, the dashboard ends up containing visualizations that rely on different data views, each with its own randomly generated ID.

https://static-www.elastic.co/v3/assets/bltefdd0b53724fa2ce/blt2a234b9b1073aa81/68409b1bd6b7cba891214b9f/count-of-records.png,count-of-records.png

These inconsistencies can cause visualizations to break when one of the associated data views is deleted, as demonstrated earlier in the Young Sheldon walkthrough.

https://static-www.elastic.co/v3/assets/bltefdd0b53724fa2ce/bltc0747f6590193050/68409bfb3010e372a4e231ab/spaces.png,spaces.pngStep 3: Document it and keep it simple

Document the naming convention and strategy in a shared knowledge repository for easy access and reference. A consistent naming approach helps keep your environment organized and makes it significantly easier to recreate or identify data views, especially when working in collaborative teams.

https://static-www.elastic.co/v3/assets/bltefdd0b53724fa2ce/blt02cb1cdfe7704811/68409c230296e44bfdb02660/custom-data-view-id-list.png,custom-data-view-id-list.png

Note: If you want to get a little fancy, you can document the strategy directly in Kibana using a text panel as shown in the image above. Alternatively, you can use a team wiki, GitHub repo, or even a simple spreadsheet (whatever works best for your workflow).