KNIME for Data Blending, ETL, Reporting & Automation

I run a regular podcast on LinkedIn Live under the title “My Data Guest”, once a month, in the middle of the month, in the middle of the week. Every episode is an interview with an expert. All my data guests are experts: Some are experts in education and careers, some are management experts, all of them are technical experts in KNIME and data science.

All these interviews are filled with expert know-how and precious practical advice. In this article, I’ve gathered what experts say about how KNIME is used in the enterprise.

Vijaykrishna Venkataram is a Senior Manager (Data Analytics) at Relevantz, a software engineering company, based in Chennai, India. His core usage of KNIME is for data blending and ETL functionalities:

“As our business is highly regulated, most of our data is hosted on-premises. However, some part of our data is scattered across multiple platforms and products. Hence, we were in need of a platform that brings everything into one place and this is where KNIME Analytics Platform came into the picture. We use KNIME’s dedicated DB nodes a lot to extract the data. What follows then is a lot of data aggregation and data cleaning - ETL basically. This makes up 70-80% of our work.”

Vijay also added automation as another prominent usage of KNIME software:

“The next step is then to automate the processes and this is where the KNIME Server (now KNIME Business Hub) demonstrates its power. Especially the possibility to connect KNIME Server with other Business Analytics tools like PowerBI is very valuable.”

Dennis Ganzaroli is Head of Report & Data-Management at Swisscom, Switzerland. He also describes the main usage of KNIME Analytics Platform for ETL, reporting, and automation:

“We use KNIME Analytics Platform mainly as an ETL tool and we also use KNIME Server (now KNIME Business Hub) to automate our workflows. We have a lot of daily reports that have to be ready in the morning, so we are happy to have such a solution. We also combine KNIME with other tools – mainly with Tableau. But I always say to the stakeholders: Tableau is just the car body, KNIME is the real engine.”

At Siemens, Philipp Kowalski, who is a digitalization evangelist, uses KNIME not just for automation, reporting, and ETL, but also adds pattern analysis to the list of KNIME qualities:

“The big advantage of using KNIME is that you learn along the way. You start with automation but while you’re applying automation you learn other tasks and nodes as well. One main usage is reporting. KNIME is a fantastic tool for reporting. Another classic usage is pattern analysis. We always wanted to know when a purchase order is likely to become problematic. To discover potential problematic orders early is of great benefit for the company as we could handle these orders with greater sensitivity. Using KNIME helped a lot in discovering these patterns.”

Tosin Adekanye works for the Qatar Financial Center Regulatory Authority, and describes herself as a Compulsive Problem Solver. What drew her to the tool is how easy it is to connect to a variety of data sources:

“I started using KNIME, because I needed something that lets me process data from all kinds of sources.”

In the interview with Evan Bristow, senior principal analyst at Genesys, data blending and ETL again show up as the most frequent usage of KNIME Analytics Platform, especially for large datasets and complex problems.

“I like to think of myself as a data MacGyver. I take different data sources or different pieces of information and pull them through an analytical process to create something that answers the question or solves the problem. Whenever the data grows big or the game gets tough, KNIME Analytics Platform is always my go-to tool. KNIME is like a Swiss army knife of data analytics and there are dozens of different ways to tackle problems, and come up with the same result.”

Evan sings KNIME’s praises for its ease of use due to the visual programming approach.

“I’m not a coder at heart so for me it’s a lot faster and more accessible to build workflows in KNIME. In my opinion, the greatest advantage is how transparent and easy to understand visual data flows are. I can always pull up a workflow, show it to a stakeholder, and we can address questions on the fly. If I were to do that in Python and write out code at the same time, I would lose business stakeholders’ attention in the blink of an eye.”

What emerges from these interviews, is the appreciation of KNIME software for its large variety of connectors and ETL operations, since both features simplify the data blending part of any data science application. A strong focus also lies on automation, as many data science applications are supposed to run at given times and to be seamlessly orchestrated together.

To summarize KNIME usage in enterprises from my interviewees covers the following areas:

1. Data blending and ETL, even for large datasets. This is mainly thanks to the wide variety of connector and data transformation nodes.

2. Reporting, either in combination with PowerBI, Tableau, or the KNIME WebPortal.

3. Productionizing data science projects with KNIME software to address automation and scalability problems in data science applications.

4. Ease of use of the tool, due to its low code visual programming approach.(This also makes sharing and documenting easier.)

Read the full interviews in the Best of KNIME booklet on KNIME Press or watch the interviews on YouTube.