Databases for Network Engineers

Understanding Databases is a must for network engineers nowadays.
We should understand databases, not only relational databases such as Mysql but also nonrelational databases (aka NOSQL), which are commonly used with Big Data.

For example we are living now in the era of Streaming Telemetry and Model Driven Telemetry (MDT).
Streaming telemetry is a new approach for network monitoring in which data is streamed from devices continuously with efficient, incremental updates.

Your Network devices such as a switch are the publisher, Your machine (where the application reside) that getting the data is the subscriber.
One of the major benefits of model-driven telemetry is that you can define the frequency and amount of data that the network device will stream back to the collector or application.

Telemetry subscription is a subscription that is used to define the set of data that is requested as part of the telemetry data.

The telemetry subscription allows you to choose the subset of the data for which you want to receive information.
There are two types of subscriptions that are used in telemetry on Cisco IOS XE Software systems:
• Dynamic (also known as dial-in) using YANG and Netconf
• Configured subscriptions (also known as dial-out) using YANG and gRPC

Switches, Routers support Streaming telemetry and can be configured to send data for instance, when CDP or BGP neighbors changed (on-change Telemetry publications); we can also get data about our switch CPU status in the last few minutes and keep getting these data every few minutes (periodic Telemetry publications)

All these data will be sent to your machine and stored in nonrelational databases from the type (Time-series database) such as InfluxDB

Another example, Cisco SD-WAN (Viptela) , all Statistics saved in your NMS (vManage) in nonrelational databases from the type (Document-based database) such as Elasticserach.
While vManage Configuration database saved in ( Graph-based database) such as Neo4j

You should know how to query these databases and how to visualize data in it using visualization tools such as Grafana or Kibana

Links:
https://developer.cisco.com/docs/ios-xe/#!streaming-telemetry-quick-start-guide
https://blogs.cisco.com/developer/getting-started-with-model-driven-telemetry
https://blogs.cisco.com/developer/model-driven-telemetry-sandbox

To download any of these databases in VM for free and practice with:
https://bitnami.com/stacks/virtual-machine

Common used nonrelational databases (aka Unstructured-Data DB)

• Key-value database: A travel blog on a website, which uses a key-value database. Each value (in this case, a blog post) is stored under a different key, and the keys represent URIs on which the blog is available.

• Document-based database: Suitable for a successful startup company that uses an application with fast-changing specifications. Data is saved as semistructured documents, facilitating change management.

• Column-based database: For business analysis, where huge amounts of data need to be processed, a column-based database is perfect because the data is stored in columns, instead of rows. Most of the operations are performed on only one column, so performance is much greater than with row-based databases.

• Time-series database: With the rise of smart cars and the Internet of Things (IoT), gigabytes of telemetry data are being generated each day. The data is sent back to a vendor for analysis and stored in a simple time-series database.

• Graph-based database: Social media has become very popular in the past few years. Due to a huge number of complex relations between entities, the social media application uses a graph-based database, where data is defined and traversed via nodes (entities) and edges (relations).