Apache NiFi: Let through the latest event in a timeframe
In the IoT world, some devices generate large volumes of events that can be difficult for back-end systems to process in real time. Of course you can use NiFi …
Category: Big Data
Apache NiFi: Monitoring metrics and provenance events using Azure Log Analytics
There are several cases where you might want to use Azure Log Analytics to monitor your NiFi instances. An obvious one is when NiFi is running in Azure. Azure …
Apache NiFi: Importing and exporting parameters
When you import a new process group or upgrade an existing one, missing parameters contexts and parameters will automatically be added. The new parameters will be filled with values …
Apache NiFi: Having fun with Jolt transformations
Jolt is a Java library which can be used to transform JSON to JSON. A Jolt transformation specification itself is also a JSON file. You can use it in …
Apache NiFi: JSON to SOAP
Apache NiFi is a powerful open source integration product. A challenge you might encounter when integrating systems is that one system can produce JSON messages and the other has …
Apache NiFi: Automating tasks using NiPyAPI
Apache NiFi has a powerful web-based interface which provides a seamless experience between design, control, feedback, and monitoring. Sometimes however, you want to automate tasks instead of doing them …
Apache NiFi: Avoid these common pitfalls
Apache NiFi is an easy to use, powerful, and reliable system to process and distribute data. It has a powerful UI which can be used for both development and …
Merge AVRO schema and generate random data or Java classes
Previously I wrote about generating random data which conforms to an AVRO schema (here). In a recent use-case, I encountered the situation where there were several separate schema files …
Apache NiFi: Forwarding HTTP headers
Apache NiFi can be used to expose various flavors of webservices. Using NiFi in such a way provides benefits like quick development using a GUI and of course data …
Apache NiFi: Reading COVID data from a REST API and producing it to a Kafka topic
Apache NiFi can be used to accelerate big data projects by allowing easy integration between various data sources. Using Apache NiFi it is easy to track what happened to …
A quick and easy Apache NiFi development environment
Vagrant can be used to quickly create development environments in for example VirtualBox, VMWare or Hyper-V. I decided to use Vagrant to create a quick Apache NiFi development environment. …
What is Apache Drill and how to setup your Proof-of-Concept
March 11, 2019
Data Analytics, Big Data, Data Warehousing & BI, Database, Databases, Microsoft Azure, NoSQL
2 Comments
Apache Drill is a schema-free SQL query engine. Drill supports a variety of NoSQL databases and file systems, including HBase, MongoDB, MapR-DB, HDFS, MapR-FS, Amazon S3, Azure Blob Storage, …
Filesystem events to Elasticsearch / Kibana through Kafka Connect / Kafka
Filesystem events are useful to monitor. They can indicate a security breach. They can also help understanding how a complex system works by looking at the files it reads …
Querying connected data in Graph databases with Neo4j
TL;DR; · Graph databases are ideal for query use cases with data with complex relationships and layers of connections · Its query language is fast, efficient and allows for …
Some impressions from Oracle Analytics Cloud–taken from keynote at Oracle OpenWorld 2017
In his keynote on October 3rd during Oracle OpenWorld 2017, Thomas Kurian stated that the vision at Oracle around analytics has changed quite considerably. He explained this change and …
The Hello World of Machine Learning – with Python, Pandas, Jupyter doing Iris classification based on quintessential set of flower data
Plenty of articles describe this hello world of Machine Learning. I will merely list some references and personal notes – primarily for my own convenience. The objective is: get …
Oracle Service Bus: A quickstart for the Kafka transport
As mentioned on the following blog post by Lucas Jellema, Kafka is going to play a part in several Oracle products. For some usecases it might eventually even replace …
The normalization of Big Data – reporting from Oracle OpenWorld 2016 on Big Data and Data Integration
The importance of data has never been in doubt in the world of Oracle. Through machine learning and predictive analytics as well as real-time streaming data and Big Data, …
Talk of the Town at Oracle OpenWorld 2016: Machine Learning & Predictive Analytics
Talk of the town during Oracle OpenWorld 2016 definitely included the term Machine Learning. Machine learning was mentioned in every other session it seemed. Sometimes fully justified – and …
Reflections after Oracle OpenWorld 2015 – Business Analytics (Big Data, GoldenGate, OBI (EE), ODI, NoSQL)
note: I would like to thank Mark Rittman of RittmanMead for sharing many of this findings from Oracle OpenWorld 2015 as well as a comprehensive slide desk. Business Analytics …
How-to bulk delete ( or archive ) as fast as possible, using minimal undo, redo and temp
Deleting some rows or tens of millions of rows from an Oracle database should be treated in a completely different fashion. Though the delete itself is technically the same, …
Gathering data for demo projects – Data Visualization, Pattern Recognition and Data Analysis based on the 2014 Eurovision Song Contest
Having access to useful data to create demonstrations and sample applications can be quite a challenge. Demonstrating the power of data visualizations (for example with ADF DVT) or the …
Oracle Database 12c: XQuery Full Text
New in Oracle 12c and one of the big new features in XMLDB is the XQuery Full Text functionality and, as mentioned in the post about XQuery Update, is …
Oracle Database 12c: XQuery Update
New, new…? No, not really new, XQuery Update (W3C standard/draft 2011) was already implemented in 11.2.0.3.0, but is now officially also announced. Besides the XQuery Full Text support (XQFT …
Hotsos Revisited 2013 – Presentatie materiaal
April 3, 2013
AMIS, Architecture, Big Data, Database, DBA Oracle, Development Tools, Oracle, Software Development, XML
No Comments
Hierbij nog dank voor allen die aanwezig waren bij de weer gevulde, informatieve & gezellige avond tijdens “Hotsos Revisited 2013”. Wij presentatoren hebben genoten van het ambiance. Hier ook …
Training Oracle ADF 11g, 15 tot en met 19 april
Van 15 tot en met 19 april geeft Luc Bors de 5-daagse ADF 11g training op het kantoor van AMIS in Nieuwegein. In 5 dagen leer je de basis …
Hotsos 2013 – Presentation material “Creating Structure in Unstructured Data”
Hereby, for those who want another look or for people to share, my presentation content “Creating Structure in Unstructured Data” given during the Hotsos 2013 Symposium on Monday morning. …
Hotsos 2013 – From Unstructured to Structured…
It has been a while that I have been attending Hotsos, although that is how it feels. In 2011 I flew to Hotsos to see, among others presentations from …
Hotsos Revisited 2013
January 28, 2013
AMIS, Architecture, Big Data, Database, DBA Oracle, Oracle, PL/SQL, XML
No Comments
Van 3 tot en met 7 maart vindt in Irving, Texas, het internationale Oracle performance Hotsos Symposium plaats. Dit jaar belooft het symposium een garantstelling voor inhoudelijk hoogstaande presentaties …
Basisregistraties Adressen en Gebouwen – Het importeren van Kadaster BAG data in een Oracle Database
Vorig jaar heb ik behoorlijk wat vragen gekregen over of er een tool was, een methodiek, om BAG data van het Nederlandse Kadaster in een Oracle database te krijgen …
Oracle XML Training With Marco Gralike
I was asked by Jože Senegačnik, if I would be would be interested in doing a Masterclass/Seminar in Slovenia and, yes of course, I really liked the idea. So …
Using the Oracle XMLDB Repository to Automatically Shred Windows Office Documents (Part 1)
December 20, 2011
AMIS, Architecture, Big Data, Database, DBA Oracle, Oracle, PL/SQL, XML
7 Comments
People who have attended the UKOUG presentation this year where Mark Drake, Sr. Product Manager XML Technologies / XMLDB, Oracle HQ, and I demonstrated the first principles of the …
UKOUG 2011: Using your Database as a Fileserver
UKOUG 2011 is nearby and one of the coolest things in Oracle 11g and onwards is, IMHO, a functionality called XDB Repository Events. Most of you probably know that …
2 dagen seminar door Steven Feuerstein: Best of Oracle PL/SQL (8 en 9 december)
October 13, 2011
Announce / Events, Architecture, Big Data, Book reviews and Publications, Cloud, Database, Databases, DBA Oracle, Development Tools, IT, Oracle, Oracle E-Business Suite, PL/SQL, SOA, Software Development, Testing, Tools
No Comments
In dit tweedaagse seminar neemt Steven Feuerstein je mee ver voorbij de basismogelijkheden van PL/SQL. Steven zal tijdens dit seminar de best practices behandelen die hij op tientallen plekken …
OOW 2011 – Oracle XMLDB and Big Data
Last day of Oracle Open World and I am currently attending the last presentations. The first presentation, “Oracle XMLDB: A noSQL Approach to Managing all your Unstructured Data”, deals …
OOW 2011 – NoSQL Databases and Oracle Database Environments
I am currently at a presentation of Patrick Schwanke, Quest Germany, regarding easy and high speed connect between NoSQL and Oracle Databases. Not really what I planned but as …