In our recipes to date, we have shown how to start up and use a single Egeria Open Metadata and Governance (OMAG) server platform running just a single OMAG server. While this is a good foundation, it is rarely useful. Egeria has many different kinds of servers, each one providing a number of services. Often, these services need to communicate with each other, even when deployed in a single JVM runtime (OMAG Server Platform). Further, we are more than likely to deploy multiple server platforms - for scale-out, availability, and many other reasons. And so the communications between servers may extend not only within a JVM, not only between servers in server platforms within a cluster, but across sites, clouds, countries and continents. So we need a mechanism that is flexible enough to support a wide range of topologies - from a single runtime to a loosely couple distributed environment. This is how we use Apache Kafka.
In this recipe we will:
Start the OMAG Server Platform with a Kafka dependent server
Install and start Kafka (if needed)
Start the environment and review our achievement
Concepts
Many of you are most likely already familiar with Apache Kafka, and many of you will be using it on a daily basis. Kafka is pretty ubiquitous and widely used across a wide variety of industries and scenarios - here is a link to their documentation Kafka Documentation. In Egeria, servers communicate via messages sent (and received) on Kafka topics. Topics are basically like a post office box that you can send mail (or Kafka message) to. To use Kafka, we specify the endpoint of a Kafka Broker to an Egeria OMAG Server's configuration. We'll cover how to do this in a future recipe. In this recipe we will start a pre-configured server that depends on Kafka - and see what happens if Kafka isn't available - as well as when it is.
1. Start the OMAG Server Platform with a Kafka dependent server
Our first step is to set up our environment to start a pre-configured OMAG server called active-metadata-store instead of simple-metadata-store. We could, of course, run both at the same time, but to keep things simple lets update the application.properties file to reflect this change - the appropriate line would look like:
startup.server.list = active-metadata-store
If we then start up the OMAG Server Platform as we have been doing with:
java -jar omag-server-platform-4.3.jar
We get a lot of output showing that the server is coming up and then at the bottom, the OMAG server tries to connect to Kafka using the endpoint it has been configured - and fails. It attempts to connect 10 times before giving up and shutting down. The output message while it is valiantly trying to connect to Kafka looks like:
Thu Jan 04 10:25:22 CST 2024 active-metadata-store Startup CONNECTED-ASSET-SERVICES-0003 The Open Connector Framework (OCF) Metadata Management Service has initialized a new instance for server active-metadata-store
Thu Jan 04 10:25:22 CST 2024 active-metadata-store Startup OPEN-METADATA-STORE-0001 The Open Metadata Store Services are initializing a new server instance
Thu Jan 04 10:25:22 CST 2024 active-metadata-store Startup OPEN-METADATA-STORE-0005 The Open Metadata Store Services has initialized a new instance for server active-metadata-store
Thu Jan 04 10:25:22 CST 2024 active-metadata-store Startup OPEN-INTEGRATION-SERVICE-0001 The Open Integration Service is initializing a new server instance
Thu Jan 04 10:25:22 CST 2024 active-metadata-store Startup OPEN-INTEGRATION-SERVICE-0005 The Open Integration Service has initialized a new instance for server active-metadata-store
Thu Jan 04 10:25:22 CST 2024 active-metadata-store Startup OMAG-ADMIN-0010 The Open Metadata Access Services (OMASs) are starting
Thu Jan 04 10:25:22 CST 2024 active-metadata-store Startup OMRS-AUDIT-0040 An enterprise OMRS connector has been created for the Data Manager OMAS
Thu Jan 04 10:25:22 CST 2024 active-metadata-store Startup OMRS-AUDIT-0041 The enterprise OMRS connector for the Data Manager OMAS has started
Thu Jan 04 10:25:22 CST 2024 active-metadata-store Startup OMAS-DATA-MANAGER-0001 The Data Manager Open Metadata Access Service (OMAS) is initializing a new server instance
Thu Jan 04 10:25:22 CST 2024 active-metadata-store Startup OCF-KAFKA-TOPIC-CONNECTOR-0001 Connecting to Apache Kafka Topic egeria.omag.server.active-metadata-store.omas.datamanager.outTopic with a server identifier of 384f4bad-aa5f-42cc-a3a8-d8ea2c43a643
Thu Jan 04 10:25:22 CST 2024 active-metadata-store Startup OCF-KAFKA-TOPIC-CONNECTOR-0015 The local server is attempting to connect to Kafka brokers at localhost:9092 [ attempt 1 of 10 ]
Thu Jan 04 10:26:22 CST 2024 active-metadata-store Startup OCF-KAFKA-TOPIC-CONNECTOR-0015 The local server is attempting to connect to Kafka brokers at localhost:9092 [ attempt 2 of 10 ]
From this output we can see that the OMAG server is expecting the Kafka endpoint to be localhost:9092. This is the endpoint that the active-metadata-store OMAG server has been configured with and Egeria is thus expecting. Port 9092 is the default port used by Kafka brokers. Next we will set up a Kafka on your local machine with this default.
2. Install and start Kafka (if needed)
There are a number of different approaches to install Kafka Apache Kafka in your environment. Choosing one depends on what operating system you are using and how much tailoring you want to do. Until recently, to install Apache Kafka, you also had to install another component called Apache Zookeeper Apache Zookeeper that Kafka depended on to provide registry and distributed coordination. Recent versions of Kafka provide an integrated option called KRaft Apache Kafka KRaft that can take over the responsibilities of Zookeeper. Some installation methods still utilize Zookeeper, others use the newer KRaft capabilities.
For the simplistic environment we need to demonstrate Egeria, any approach will work as long as we are able to establish a Kafka endpoint at localhost:9092. For more permanent deployments, either the choices will be determined by an infrastructure team or there is a bit of investigation and design work to perform as part of an overall Egeria deployment design.
There are many available tutorials and guides for how to install Kafka in different environments - so rather than replicate that in this recipe, we'll describe one simple approach on a MacOS; if your environment is different and you need a pointer, you can reach out to us on the Egeria Community Slack Channel or send us an Email Egeria Community Guide.
For this installation we will use HomeBrew, a package manager available on MacOS and Linux. If you need to install Homebrew, you can follow the instructions at Homebrew; Homebrew is a commonly used way to install useful packages on MacOS. Once Homebrew is installed and running, we can install Apache Zookeeper and then Apache Kafka - it would look something like this:
➜brew install zookeeper
Running `brew update --auto-update`...
==> Auto-updated Homebrew!
Updated 2 taps (homebrew/core and homebrew/cask).
==> Downloading https://ghcr.io/v2/homebrew/core/zookeeper/manifests/3.9.1
######################################################################### 100.0%
==> Fetching zookeeper
==> Downloading https://ghcr.io/v2/homebrew/core/zookeeper/blobs/sha256:62bdde8b
######################################################################### 100.0%
==> Pouring zookeeper--3.9.1.sonoma.bottle.tar.gz
==> Downloading https://raw.githubusercontent.com/apache/zookeeper/release-3.9.0
######################################################################### 100.0%
==> Caveats
To start zookeeper now and restart at login:
brew services start zookeeper
Or, if you don't want/need a background service you can just run:
SERVER_JVMFLAGS="-Dapple.awt.UIElement=true" /usr/local/opt/zookeeper/bin/zkServer start-foreground
==> Summary
🍺 /usr/local/Cellar/zookeeper/3.9.1: 1,115 files, 62.7MB
==> Running `brew cleanup zookeeper`...
Disable this behaviour by setting HOMEBREW_NO_INSTALL_CLEANUP.
Hide these hints with HOMEBREW_NO_ENV_HINTS (see `man brew`).
Removing: /Users/dwolfson/Library/Caches/Homebrew/zookeeper--default_logback_xml--3.9.0.xml... (4.5KB)
➜ brew install kafka
==> Downloading https://ghcr.io/v2/homebrew/core/kafka/manifests/3.6.1
Already downloaded: /Users/dwolfson/Library/Caches/Homebrew/downloads/1b91a3663cc141c54ba511531dac0335bd48bd95666b2b91ab2a2656b5ebc092--kafka-3.6.1.bottle_manifest.json
==> Fetching kafka
==> Downloading https://ghcr.io/v2/homebrew/core/kafka/blobs/sha256:b2b11192671b
Already downloaded: /Users/dwolfson/Library/Caches/Homebrew/downloads/ca896b04f3df8ab514f23ab59349dc6644256640615d6c6a3ecb16e0d343a66c--kafka--3.6.1.sonoma.bottle.tar.gz
==> Pouring kafka--3.6.1.sonoma.bottle.tar.gz
==> Caveats
To start kafka now and restart at login:
brew services start kafka
Or, if you don't want/need a background service you can just run:
/usr/local/opt/kafka/bin/kafka-server-start /usr/local/etc/kafka/server.properties
==> Summary
🍺 /usr/local/Cellar/kafka/3.6.1: 222 files, 108.9MB
==> Running `brew cleanup kafka`...
Disable this behaviour by setting HOMEBREW_NO_INSTALL_CLEANUP.
Hide these hints with HOMEBREW_NO_ENV_HINTS (see `man brew`).
At this point we have both Zookeeper and Kafka installed - but they aren't running yet. We can use the default configurations that brew installs them with - although the configurations can be customized later. Now we need to start both Zookeeper and Kafka - the text above told us how, but to be clear we issue:
brew services start zookeeper
and then:
brew services start kafka
You should now have both started. We can check by issueing the command:
brew services list
Which tells us that both of the services have been started. As you might surmise, to stop, for instance, the kafka service we type:
brew services stop kafka
OK, now that we have Kafka (and Zookeeper) running we can move on to start the OMAG Server Platform.
3. Start the environment and review our achievement
Lets now start the OMAG Server Platform as we did in step 1 by issuing:
java -jar omag-server-platform-4.3.jar
Wow! The output looks quite different this time! Here is what my output looks like (well - the last ~70 lines or so):
Thu Jan 04 19:56:47 CST 2024 active-metadata-store Startup OCF-KAFKA-TOPIC-CONNECTOR-0001 Connecting to Apache Kafka Topic egeria.omag.server.active-metadata-store.omas.assetlineage.outTopic with a server identifier of 384f4bad-aa5f-42cc-a3a8-d8ea2c43a643
Thu Jan 04 19:56:47 CST 2024 active-metadata-store Startup OCF-KAFKA-TOPIC-CONNECTOR-0015 The local server is attempting to connect to Kafka brokers at localhost:9092 [ attempt 1 of 10 ]
Thu Jan 04 19:56:47 CST 2024 active-metadata-store Startup OCF-KAFKA-TOPIC-CONNECTOR-0003 10 properties passed to the Apache Kafka Consumer for topic egeria.omag.server.active-metadata-store.omas.assetlineage.outTopic
Thu Jan 04 19:56:47 CST 2024 active-metadata-store Startup OCF-KAFKA-TOPIC-CONNECTOR-0002 10 properties passed to the Apache Kafka Producer for topic egeria.omag.server.active-metadata-store.omas.assetlineage.outTopic
Thu Jan 04 19:56:47 CST 2024 active-metadata-store Startup OCF-KAFKA-TOPIC-CONNECTOR-0010 The Apache Kafka producer for topic egeria.omag.server.active-metadata-store.omas.assetlineage.outTopic is starting up with 0 buffered messages
Thu Jan 04 19:56:47 CST 2024 active-metadata-store Startup OMRS-AUDIT-0015 The listener thread for an OMRS Topic Connector for topic egeria.omag.server.active-metadata-store.omas.assetlineage.outTopic has started
Thu Jan 04 19:56:47 CST 2024 active-metadata-store Startup OMAG-ADMIN-0209 The Asset Lineage Open Metadata Access Service (OMAS) is registering a listener with the enterprise OMRS Topic for server active-metadata-store
Thu Jan 04 19:56:47 CST 2024 active-metadata-store Startup OMAS-ASSET-LINEAGE-0002 The Asset Lineage Open Metadata Access Service (OMAS) has initialized a new instance for server active-metadata-store
Thu Jan 04 19:56:47 CST 2024 active-metadata-store Startup OMRS-AUDIT-0040 An enterprise OMRS connector has been created for the Asset Consumer OMAS
Thu Jan 04 19:56:47 CST 2024 active-metadata-store Startup OMRS-AUDIT-0041 The enterprise OMRS connector for the Asset Consumer OMAS has started
Thu Jan 04 19:56:47 CST 2024 active-metadata-store Startup OMAS-ASSET-CONSUMER-0001 The Asset Consumer Open Metadata Access Service (OMAS) is initializing a new server instance
Thu Jan 04 19:56:47 CST 2024 active-metadata-store Startup OCF-KAFKA-TOPIC-CONNECTOR-0001 Connecting to Apache Kafka Topic egeria.omag.server.active-metadata-store.omas.assetconsumer.outTopic with a server identifier of 384f4bad-aa5f-42cc-a3a8-d8ea2c43a643
Thu Jan 04 19:56:47 CST 2024 active-metadata-store Startup OCF-KAFKA-TOPIC-CONNECTOR-0015 The local server is attempting to connect to Kafka brokers at localhost:9092 [ attempt 1 of 10 ]
Thu Jan 04 19:56:47 CST 2024 active-metadata-store Startup OCF-KAFKA-TOPIC-CONNECTOR-0002 10 properties passed to the Apache Kafka Producer for topic egeria.omag.server.active-metadata-store.omas.assetconsumer.outTopic
Thu Jan 04 19:56:47 CST 2024 active-metadata-store Startup OCF-KAFKA-TOPIC-CONNECTOR-0010 The Apache Kafka producer for topic egeria.omag.server.active-metadata-store.omas.assetconsumer.outTopic is starting up with 0 buffered messages
Thu Jan 04 19:56:47 CST 2024 active-metadata-store Startup OMRS-AUDIT-0015 The listener thread for an OMRS Topic Connector for topic egeria.omag.server.active-metadata-store.omas.assetconsumer.outTopic has started
Thu Jan 04 19:56:47 CST 2024 active-metadata-store Startup OMAS-ASSET-CONSUMER-0002 The Asset Consumer Open Metadata Access Service (OMAS) is ready to publish asset notifications to topic egeria.omag.server.active-metadata-store.omas.assetconsumer.outTopic
Thu Jan 04 19:56:47 CST 2024 active-metadata-store Startup OMAG-ADMIN-0209 The Asset Consumer OMAS Open Metadata Access Service (OMAS) is registering a listener with the enterprise OMRS Topic for server active-metadata-store
Thu Jan 04 19:56:47 CST 2024 active-metadata-store Startup OMAS-ASSET-CONSUMER-0003 The Asset Consumer Open Metadata Access Service (OMAS) has initialized a new instance for server active-metadata-store
Thu Jan 04 19:56:47 CST 2024 active-metadata-store Startup OMRS-AUDIT-0040 An enterprise OMRS connector has been created for the Asset Catalog OMAS
Thu Jan 04 19:56:47 CST 2024 active-metadata-store Startup OMRS-AUDIT-0041 The enterprise OMRS connector for the Asset Catalog OMAS has started
Thu Jan 04 19:56:47 CST 2024 active-metadata-store Startup OMAS-ASSET-CATALOG-0002 The Asset Catalog Open Metadata Access Service (OMAS) is initializing a new server instance
Thu Jan 04 19:56:47 CST 2024 active-metadata-store Startup OMAS-ASSET-CATALOG-0001 The Asset Catalog Open Metadata Access Service (OMAS) has initialized a new instance for server active-metadata-store
Thu Jan 04 19:56:47 CST 2024 active-metadata-store Startup OMRS-AUDIT-0040 An enterprise OMRS connector has been created for the DevOps OMAS
Thu Jan 04 19:56:47 CST 2024 active-metadata-store Startup OMRS-AUDIT-0041 The enterprise OMRS connector for the DevOps OMAS has started
Thu Jan 04 19:56:47 CST 2024 active-metadata-store Startup OMAS-DEV-OPS-0001 The DevOps Open Metadata Access Service (OMAS) is initializing a new server instance
Thu Jan 04 19:56:47 CST 2024 active-metadata-store Startup OMAS-DEV-OPS-0003 The DevOps Open Metadata Access Service (OMAS) has initialized a new instance for server active-metadata-store
Thu Jan 04 19:56:47 CST 2024 active-metadata-store Startup OMRS-AUDIT-0040 An enterprise OMRS connector has been created for the Software Developer OMAS
Thu Jan 04 19:56:47 CST 2024 active-metadata-store Startup OMRS-AUDIT-0041 The enterprise OMRS connector for the Software Developer OMAS has started
Thu Jan 04 19:56:47 CST 2024 active-metadata-store Startup OMAS-SOFTWARE-DEVELOPER-0001 The Software Developer Open Metadata Access Service (OMAS) is initializing a new server instance
Thu Jan 04 19:56:47 CST 2024 active-metadata-store Startup OMAS-SOFTWARE-DEVELOPER-0003 The Software Developer Open Metadata Access Service (OMAS) has initialized a new instance for server active-metadata-store
Thu Jan 04 19:56:47 CST 2024 active-metadata-store Startup OMRS-AUDIT-0040 An enterprise OMRS connector has been created for the Project Management OMAS
Thu Jan 04 19:56:47 CST 2024 active-metadata-store Startup OMRS-AUDIT-0041 The enterprise OMRS connector for the Project Management OMAS has started
Thu Jan 04 19:56:47 CST 2024 active-metadata-store Startup OMAS-PROJECT-MANAGEMENT-0001 The Project Management Open Metadata Access Service (OMAS) is initializing a new server instance
Thu Jan 04 19:56:47 CST 2024 active-metadata-store Startup OMAS-PROJECT-MANAGEMENT-0003 The Project Management Open Metadata Access Service (OMAS) has initialized a new instance for server active-metadata-store
Thu Jan 04 19:56:47 CST 2024 active-metadata-store Startup OMRS-AUDIT-0040 An enterprise OMRS connector has been created for the Governance Engine OMAS
Thu Jan 04 19:56:47 CST 2024 active-metadata-store Startup OMRS-AUDIT-0041 The enterprise OMRS connector for the Governance Engine OMAS has started
Thu Jan 04 19:56:47 CST 2024 active-metadata-store Startup OMAS-GOVERNANCE-ENGINE-0001 The Governance Engine Open Metadata Access Service (OMAS) is initializing a new server instance
Thu Jan 04 19:56:47 CST 2024 active-metadata-store Startup OCF-KAFKA-TOPIC-CONNECTOR-0001 Connecting to Apache Kafka Topic egeria.omag.server.active-metadata-store.omas.governanceengine.outTopic with a server identifier of 384f4bad-aa5f-42cc-a3a8-d8ea2c43a643
Thu Jan 04 19:56:47 CST 2024 active-metadata-store Startup OCF-KAFKA-TOPIC-CONNECTOR-0015 The local server is attempting to connect to Kafka brokers at localhost:9092 [ attempt 1 of 10 ]
Thu Jan 04 19:56:47 CST 2024 active-metadata-store Startup OCF-KAFKA-TOPIC-CONNECTOR-0002 10 properties passed to the Apache Kafka Producer for topic egeria.omag.server.active-metadata-store.omas.governanceengine.outTopic
Thu Jan 04 19:56:47 CST 2024 active-metadata-store Startup OCF-KAFKA-TOPIC-CONNECTOR-0010 The Apache Kafka producer for topic egeria.omag.server.active-metadata-store.omas.governanceengine.outTopic is starting up with 0 buffered messages
Thu Jan 04 19:56:47 CST 2024 active-metadata-store Startup OMRS-AUDIT-0015 The listener thread for an OMRS Topic Connector for topic egeria.omag.server.active-metadata-store.omas.governanceengine.outTopic has started
Thu Jan 04 19:56:47 CST 2024 active-metadata-store Startup OMAS-GOVERNANCE-ENGINE-0004 The Governance Engine Open Metadata Access Service (OMAS) is ready to publish configuration refresh notifications to topic egeria.omag.server.active-metadata-store.omas.governanceengine.outTopic
Thu Jan 04 19:56:47 CST 2024 active-metadata-store Startup OMAG-ADMIN-0209 The Governance Engine OMAS Open Metadata Access Service (OMAS) is registering a listener with the enterprise OMRS Topic for server active-metadata-store
Thu Jan 04 19:56:47 CST 2024 active-metadata-store Startup OMAS-GOVERNANCE-ENGINE-0005 The Governance Engine Open Metadata Access Service (OMAS) has initialized a new instance for server active-metadata-store
Thu Jan 04 19:56:47 CST 2024 active-metadata-store Startup OMRS-AUDIT-0040 An enterprise OMRS connector has been created for the Data Privacy OMAS
Thu Jan 04 19:56:47 CST 2024 active-metadata-store Startup OMRS-AUDIT-0041 The enterprise OMRS connector for the Data Privacy OMAS has started
Thu Jan 04 19:56:47 CST 2024 active-metadata-store Startup OMAS-DATA-PRIVACY-0001 The Data Privacy Open Metadata Access Service (OMAS) is initializing a new server instance
Thu Jan 04 19:56:47 CST 2024 active-metadata-store Startup OMAS-DATA-PRIVACY-0003 The Data Privacy Open Metadata Access Service (OMAS) has initialized a new instance for server active-metadata-store
Thu Jan 04 19:56:47 CST 2024 active-metadata-store Startup OMRS-AUDIT-0040 An enterprise OMRS connector has been created for the Security Manager OMAS
Thu Jan 04 19:56:47 CST 2024 active-metadata-store Startup OMRS-AUDIT-0041 The enterprise OMRS connector for the Security Manager OMAS has started
Thu Jan 04 19:56:47 CST 2024 active-metadata-store Startup OMAS-SECURITY-MANAGER-0001 The Security Manager Open Metadata Access Service (OMAS) is initializing a new server instance
Thu Jan 04 19:56:47 CST 2024 active-metadata-store Startup OCF-KAFKA-TOPIC-CONNECTOR-0001 Connecting to Apache Kafka Topic egeria.omag.server.active-metadata-store.omas.securitymanager.outTopic with a server identifier of 384f4bad-aa5f-42cc-a3a8-d8ea2c43a643
Thu Jan 04 19:56:47 CST 2024 active-metadata-store Startup OCF-KAFKA-TOPIC-CONNECTOR-0015 The local server is attempting to connect to Kafka brokers at localhost:9092 [ attempt 1 of 10 ]
Thu Jan 04 19:56:47 CST 2024 active-metadata-store Startup OCF-KAFKA-TOPIC-CONNECTOR-0002 10 properties passed to the Apache Kafka Producer for topic egeria.omag.server.active-metadata-store.omas.securitymanager.outTopic
Thu Jan 04 19:56:47 CST 2024 active-metadata-store Startup OCF-KAFKA-TOPIC-CONNECTOR-0010 The Apache Kafka producer for topic egeria.omag.server.active-metadata-store.omas.securitymanager.outTopic is starting up with 0 buffered messages
Thu Jan 04 19:56:47 CST 2024 active-metadata-store Startup OMRS-AUDIT-0015 The listener thread for an OMRS Topic Connector for topic egeria.omag.server.active-metadata-store.omas.securitymanager.outTopic has started
Thu Jan 04 19:56:47 CST 2024 active-metadata-store Startup OMAS-SECURITY-MANAGER-0002 The Security Manager Open Metadata Access Service (OMAS) is ready to publish security manager notifications to topic egeria.omag.server.active-metadata-store.omas.securitymanager.outTopic
Thu Jan 04 19:56:47 CST 2024 active-metadata-store Startup OMAG-ADMIN-0209 The Security Manager OMAS Open Metadata Access Service (OMAS) is registering a listener with the enterprise OMRS Topic for server active-metadata-store
Thu Jan 04 19:56:47 CST 2024 active-metadata-store Startup OMAS-SECURITY-MANAGER-0005 The Security Manager Open Metadata Access Service (OMAS) has initialized a new instance for server active-metadata-store
Thu Jan 04 19:56:47 CST 2024 active-metadata-store Startup OMAG-ADMIN-0012 25 out of 25 configured Open Metadata Access Services (OMASs) have started.
Thu Jan 04 19:56:47 CST 2024 active-metadata-store Startup OMRS-AUDIT-0019 An OMRS Topic Connector has registered with an event bus connector for topic active-metadata-store.openmetadata.repositoryservices.enterprise.active-metadata-store.OMRSTopic
Thu Jan 04 19:56:47 CST 2024 active-metadata-store Startup OMRS-AUDIT-0020 An OMRS Topic Connector is ready to send and receive events on topic active-metadata-store.openmetadata.repositoryservices.enterprise.active-metadata-store.OMRSTopic
Thu Jan 04 19:56:47 CST 2024 active-metadata-store Startup OMRS-AUDIT-0015 The listener thread for an OMRS Topic Connector for topic active-metadata-store.openmetadata.repositoryservices.enterprise.active-metadata-store.OMRSTopic has started
Thu Jan 04 19:56:47 CST 2024 active-metadata-store Startup OMAG-ADMIN-0004 The active-metadata-store server has successfully completed start up. The following services are running: [Open Metadata Repository Services (OMRS), Connected Asset Services, Governance Action Framework Services, Open Integration Service, Data Manager OMAS, Subject Area OMAS, Glossary View OMAS, Design Model OMAS, Security Officer OMAS, Asset Manager OMAS, IT Infrastructure OMAS, Data Science OMAS, Community Profile OMAS, Data Engine OMAS, Discovery Engine OMAS, Digital Architecture OMAS, Asset Owner OMAS, Stewardship Action OMAS, Governance Program OMAS, Digital Service OMAS, Asset Lineage OMAS, Asset Consumer OMAS, Asset Catalog OMAS, DevOps OMAS, Software Developer OMAS, Project Management OMAS, Governance Engine OMAS, Data Privacy OMAS, Security Manager OMAS]
Thu Jan 04 19:56:47 CST 2024 OMAG Server 'active-metadata-store' successful start , with message: Thu Jan 04 19:56:47 CST 2024 active-metadata-store is running the following services: [Open Metadata Repository Services (OMRS), Connected Asset Services, Governance Action Framework Services, Open Integration Service, Data Manager OMAS, Subject Area OMAS, Glossary View OMAS, Design Model OMAS, Security Officer OMAS, Asset Manager OMAS, IT Infrastructure OMAS, Data Science OMAS, Community Profile OMAS, Data Engine OMAS, Discovery Engine OMAS, Digital Architecture OMAS, Asset Owner OMAS, Stewardship Action OMAS, Governance Program OMAS, Digital Service OMAS, Asset Lineage OMAS, Asset Consumer OMAS, Asset Catalog OMAS, DevOps OMAS, Software Developer OMAS, Project Management OMAS, Governance Engine OMAS, Data Privacy OMAS, Security Manager OMAS]
Thu Jan 04 19:56:47 CST 2024 OMAG server platform ready for more configuration
If you peruse through the output we can see the machinery of Egeria kick into gear and startup different services and connectors within the active-metadata-store OMAG server. Towards the end it tells us all of the servers that have been configured and that they have been successfully started. And now the OMAG server platform is ready for more work! We'll try out some of these services in the next recipe.
But before we begin to wrap-up, it is worth taking a brief look at what we have now configured by re-running the Platform Report that we used in recipe 2. To do this we open a new terminal window and issue the following commands:
Change directory to the reports directory - in my case:
cd ~/egeria_sandbox/egeria-platform-4.3/sandbox.gz/assembly/etc/reports
Issue:
java -jar egeria-platform-report.jar
Review: I find its easiest to look at the markdown file egeria-platform-report.md with an editor that understands markdown format.
Now what you see is a somewhat simpler, but much more readable configuration report compared to what comes out of the console. It lists the servers, services and connectors that are running on the OMAG server platform. As you can see there is a rich set of capabilities to explore - and in our next recipe we will start to do just that.
Recap
In this recipe we learned how Egeria uses Kafka to communicate and coordinate both among Egeria's services and potentially with other runtimes in what we broadly call the Open Metadata Ecosystem. We will explore the ecosystem more in a future recipe.
We showed how Egeria complains if we try to start up an OMAG server that depends on Kafka - and it can't be found; and then how we can install Kafka, restart the OMAG server platform and complete this recipe with a happy Egeria OMAG server with services that are ready and just waiting to be used. So that is what we will do in our next episode - stay tuned!