AMIS, Data Driven Blog – Oracle & Microsoft Azure
Getting value from IoT, Integration and Data Analytics
There are several cases where you might want to use Azure Log Analytics to monitor your NiFi instances. An obvious one is when NiFi is running in Azure. Azure Log Analytics can also be used as single monitoring/alerting solution for multiple applications making operations easier by providing a single interface […]
0 
0 When you import a new process group or upgrade an existing one, missing parameters contexts and parameters will automatically be added. The new parameters will be filled with values from the environment where the process group was committed to the Registry (except sensitive parameter values). This is usually a development […]
0 0 Jolt is a Java library which can be used to transform JSON to JSON. A Jolt transformation specification itself is also a JSON file. You can use it in products such as Apache NiFi and Apache Camel. In this blog post I’ll describe my first experiences with Jolt transformations. For […]
0 0 Apache NiFi is a powerful open source integration product. A challenge you might encounter when integrating systems is that one system can produce JSON messages and the other has a SOAP API available. In this blog post I’ll show how you can use NiFi to convert JSON input to a […]
0 0 Apache NiFi has a powerful web-based interface which provides a seamless experience between design, control, feedback, and monitoring. Sometimes however, you want to automate tasks instead of doing them manually using the UI. This does not only allow you to perform the tasks a lot quicker but it also helps […]
0 0 Apache NiFi is an easy to use, powerful, and reliable system to process and distribute data. It has a powerful UI which can be used for both development and operations. In addition, the NiFi Registry is available to make promoting software from one environment to the next, easy. In order […]
0 0 I’ve used Vagrant since 2015 in combination with Virtualbox for creating development machines. Recently however I’m experiencing more issues with Virtualbox. For example CPUs getting stuck when assigning multiple CPUs to a VM and issues with auto adjusting the guest resolution when resizing the VM window. These annoyances drove me […]
0 0 Previously I wrote about generating random data which conforms to an AVRO schema (here). In a recent use-case, I encountered the situation where there were several separate schema files containing different AVRO types. The message used types from those different files. For the generation of random data, I first needed […]
0 0 AVRO schema are often used to serialize JSON data into a compact binary format in order to for example transport it efficiently over Kafka. When you want to validate your JSON against an AVRO schema in Java, you will encounter some challenges. The JSON which is required to allow validation […]
0 0 Recently I was designing an AVRO schema and wanted to test how data would look like which conformed to this schema. I developed some Java code to generate sample data. This of course also has uses in more elaborate tests which require generation of random events. Because AVRO is not […]
0 0 Apache NiFi can be used to expose various flavors of webservices. Using NiFi in such a way provides benefits like quick development using a GUI and of course data provenance. You know who called you with which data and where the data went. The NiFi is very scalable, delivery can […]
0 0 Apache NiFi can be used to accelerate big data projects by allowing easy integration between various data sources. Using Apache NiFi it is easy to track what happened to your data (data provenance) and to provide features like guaranteed ordered delivery and error handling. In this example I’m going to […]
0 0 Vagrant can be used to quickly create development environments in for example VirtualBox, VMWare or Hyper-V. I decided to use Vagrant to create a quick Apache NiFi development environment. For Apache NiFi development, you also often require input/output for which Kafka can be used, the NiFi Registry to manage shared […]
0 0 Your customer has a specific challenge and wants to have it solved by means of software. They ask you as a software/solution architect to advise them on this topic and they want to be able to choose a solution based on your advise by the end of next week (we’re […]
0 0 Quality is something which is hard to define. Different areas of expertise use their own definitions of what quality is. Without an objective standard which carries weight, anyone can claim to provide a quality product or service according to some standard. This makes it difficult to compare products and to […]
0 0 Sometimes you want to record online presentations to maybe re-watch them later or share them with colleagues. It is not always possible to easily do this from within the tool which is used for the presentation. An easy alternative is to record such sessions using OBS Studio, a free and […]
0 0 This is my 200th blog post. Also I reached 1M page-views recently. In those 200 blog posts, I used approximately 325K words. An average non-fiction book is somewhere between 70K to 80K words. So that would amount to somewhere around 4 books. Why did I write so many blog posts, […]
0 0 Jenkins is a solid CI/CD platform which has proven itself over the years. Many organizations use it to build, test and deploy their applications. In Jenkins it is possible to define credentials or to use an external credential store. You can then use these inside your pipelines and jobs. Direct […]
0 0 SonarCloud is a hosted SonarQube SaaS solution which helps you with code quality management. It is free to use for open source projects. You cannot install 3rd party plugins in SonarCloud however. This puts some limitations on the kind of data you can put in SonarCloud. For Java this is […]
0 0 GitHub Actions allow you to do most CI/CD tasks for free, directly from your GitHub repository. One of the challenges however is that there is no build-in facility like for example SonarQube to manage code quality. Luckily, SonarSource provides SonarCloud; a SonarQube SaaS offering which is free for public projects! […]
0 0 I’m a regular user of GitHub. Recently I discovered GitHub also has a build-in CI/CD workflow solution called GitHub Actions. Curious about how this would work I decided to try it out. I had previously build a Jenkins Pipeline to perform several static and dynamic application security tests on a Java project […]
0 0 OWASP ZAP (Zed Attack Proxy) is an open source dynamic application security testing (DAST) tool. It is available here and has a website with documentation here. I recently encountered it when looking for open source security test tools to embed in a CI/CD pipeline (here). I was surprised by how versatile this tool […]
0 0 It is often expected of a DevOps team to also take security into consideration when delivering software. Often however, this does not get the attention it deserves. In this blog post I’ll describe some easy to use, CI/CD pipeline friendly, open source tools you can use to perform several checks […]
0 0 You might want to change the behavior of a Java application at runtime without having to alter the original sources and recompile them. This can be done by using a Java Agent. Java Agents are used by several products. Dynatrace uses for example a Java Agent to collect data from inside […]
0 0 There are a lot of use-cases in which you might want to automate web-browser actions. For example to automate tedious repetitive tasks or to perform automated tests of front-end applications. There are several tools available to do this such as Selenium, Cypress and Puppeteer. Several blog posts (such as here) […]
0 0 The OWASP Foundation plays an important role in helping to improve security of software worldwide. They have created a popular and well-known awareness document called the ‘OWASP Top 10‘. This document lists the following risk: using components with known vulnerabilities. Software nowadays can be quite complex consisting of many direct and […]
0 0 Anchore Engine is a popular open source tool for container image inspection and vulnerability scanning. It is easily integrated in a Kubernetes environment as an admission controller or in a Jenkins build pipeline using a plugin. A while ago I took a look at Anchore Engine and created a small introductory presentation and Katacoda scenario for […]
0 0 Kubernetes is a popular platform to run and manage containerized applications. A CI/CD solution is often needed but not always provided. You might need to set this up for yourself. In this blog post I’ll provide a minimal end-to-end solution for Java applications. This starts with a commit in source […]
0 0 OpenEBS provides a Kubernetes native distributed storage solution which is friendly on developers and administrators. It is completely open source and part of the CNCF. Previously I wrote about installing and using OpenEBS, Jiva storage engine, on the Charmed Kubernetes distribution of Canonical. The Jiva storage class uses storage inside managed […]
0 0 There are various options to install a production-like Kubernetes distribution on your laptop. Previously I tried out using the Canonical stack (Juju, MAAS, Charmed Kubernetes) for this. This worked nicely but it gave me the feeling that it was a bit Canonical specific and it felt a bit heavy on resources at […]
0 0 Applications nowadays, are usually deployed inside containers. A container consists of libraries and tools which allow the application to run inside. Since there can be exploitable vulnerabilities, it is not only important to take security into account for your application, but also for the container it runs in. There are […]
0 0 When giving online presentations, it helps for personal marketing if people can see you on the screen. Various tools provide features which help you achieve that, for example Microsoft Teams. Sometimes though you do not have that available or want to be able to do more than what the tool […]
0 0 As a developer I wanted to experiment with Kubernetes environments which approximate production deployments. In order to do that I wanted a distributed storage solution and chose OpenEBS. Mainly because it was easy to get started and quick to get up and running. In this blog post I’ll describe how […]
0 0 Coming to this solution was a journey. I was looking for a Kubernetes installation which was easy to deploy and cleanup on my own laptop (I didn’t want to have to pay for a hosted solution). I did want a solution which was more or less production-like because I wanted […]
0 0 Kubernetes is a popular container orchestration platform. As a developer understanding the environment in which your application is going to run is important since this can help you use available services of the platform and fix issues. There are several options to run Kubernetes locally to get some experience with […]
0 0 Spring Framework version 5, released in Sept 2017, introduced Spring WebFlux. A fully reactive stack. In Dec 2019 Spring Data R2DBC was released, an incubator to integrate relational databases using a reactive driver. In this blog post I’ll show that at high concurrency, WebFlux and R2DBC perform better. They have […]
0 0 When running Java applications in containers, you need to be careful with your resources. If you’re not careful with layering your images (for example using Google’s Jib), you can quickly get into disk-space issues, especially when your base image and/or application gets updated regularly. One of the ways you can save […]
0 0 R2DBC provides non-blocking reactive APIs to relational database programmers in Java. It is an open specification, similar to JDBC. JDBC however uses a thread per connection while R2DBC can handle more connections using less threads (and thus potentially use less memory). This could also mean threads are available to do other […]
0 0 Oracle provides several services as ‘always free’. In contrast to Azure and Amazon, these include compute instances which remain ‘forever’ free to use. Although there are some limitations on CPU, disk, network resources, these instances are ideal to use as a remote SSH server and with a little effort a […]
0 0 wrk is a modern HTTP benchmarking tool. Using a simple CLI interface you can put simple load on HTTP services and determine latency, response times and the number of successfully processed requests. It has a LuaJIT scripting interface which provides extensibility. A distinguishing feature of wrk compared to for example ab […]
1 0 
