AMIS, Data Driven Blog – Oracle & Microsoft Azure
Getting value from IoT, Integration and Data Analytics
AVRO schema are often used to serialize JSON data into a compact binary format in order to for example transport it efficiently over Kafka. When you want to validate your JSON against an AVRO schema in Java, you will encounter some challenges. The JSON which is required to allow validation […]
0 
0 Recently I was designing an AVRO schema and wanted to test how data would look like which conformed to this schema. I developed some Java code to generate sample data. This of course also has uses in more elaborate tests which require generation of random events. Because AVRO is not […]
0 0 I wanted to set up a demo environment with Apache Kafka (an open-source distributed event streaming platform) and an Oracle Database, all within containers. For my purpose I opted for Oracle Database XE. [https://kafka.apache.org/] In a series of articles, I will take you with me on my path to set […]
0 0 I wanted to set up a demo environment with Apache Kafka (an open-source distributed event streaming platform) and an Oracle Database, all within containers. For my purpose I opted for Oracle Database XE. [https://kafka.apache.org/] In my previous article, I described the steps I took, to set up a demo environment […]
0 0 Vagrant can be used to quickly create development environments in for example VirtualBox, VMWare or Hyper-V. I decided to use Vagrant to create a quick Apache NiFi development environment. For Apache NiFi development, you also often require input/output for which Kafka can be used, the NiFi Registry to manage shared […]
0 0 Last week I presented on Apache Kafka – twice. Once to a group of over 100 students, once to 30+ colleagues. In both instances, I invited attendees to partake in a workshop with hands-on labs to get acquainted with Apache Kafka. I had prepared a Docker Compose based Kafka platform […]
0 0 Filesystem events are useful to monitor. They can indicate a security breach. They can also help understanding how a complex system works by looking at the files it reads and writes. When monitoring events, you can expect a lot of data to be generated quickly. The events might be interesting […]
0 0 For conducting some experiments and preparing several demonstrations I needed a locally running Kafka Cluster (of a recent release) in combination with a KSQL server instance. Additional components from the Core Kafka Project and the Confluent Open Source Platform (release 4.1) would be convenient to have. I needed everything to […]
0 0 Put simply – CQRS (Command Query Responsibility Segregation) is an architecture pattern that recognizes the fact that it may be wise to separate the database that processes data manipulations from the engines that handle queries. When data retrieval requires special formats, scale, availability, TCO, location, search options and response times, […]
0 0 This article describes how to quickly get going with KSQL on Kubernetes. KSQL is Confuent’s ‘continuous streaming query language’. It allows us to write SQL-like queries that operate on Kafka Topics. Queries that join, filter and aggregate – for each event that gets produced and over time windows. Results are […]
0 0 For workshop I will present on microservices and communication patterns I need attendees to have their own local Kafka Cluster. I have found a way to have them up and running in virtually no time at all. Thanks to the combination of: Kubernetes Minikube The Yolean/kubernetes-kafka GitHub Repo with Kubernetes […]
0 0 On YouTube, you will find the 44 min recording of my talk on Oracle Code San Francisco (October 2017): https://www.youtube.com/watch?v=khP3yHmbtpM The slides are on SlideShare: Real Time UI with Apache Kafka Streaming Analytics of Fast Data and Server Push (OracleCode San Francisco 2017) from Lucas Jellema All code sources are […]
0 0 In several previous articles on Apache Kafka, Kafka Streams and Node.JS for interacting with Apache Kafka, I have described how to create a Node.JS application that publishes messages to a Kafka Topic (based on entries in a CSV file), how to create a simple Kafka Streams Java application that processes […]
0 0 Oracle Service Bus is a powerful tool to provide features like transformation, throttling, virtualization of messages coming from different sources. There is a (recently opensourced!) Kafka transport available for Oracle Service Bus (see here). Oracle Service Bus can thus be used to do all kinds of interesting things to messages […]
0 0 
