KNIME logo
Contact usDownload
KNIME logo
Blog
Read time: 3 min

How to Use Python Script Node in Production Environments

Execute Workflows with Python Scripts on the Fly: No Custom Python Installation Required!

December 8, 2022
Scripting
python-script-node-v-4-7.jpg
Stacked TrianglesPanel BG

The need for low-code tools has enabled citizen data scientists to create efficient and reusable data science solutions. As a result, low-code solutions have gained a lot of attention, plus they are easy to debug and interpret. With the KNIME-Python Integration, we enable the users to combine the best of visual programming with the scripting world.

With KNIME Analytics Platform 4.7 the Python Script node is out of Labs and part of the production-ready KNIME Python Integration.

2022-11-30_08h53_53.gif
Figure 1: Python Script Node and its configuration dialogue that shows Input tables or variables, the editor to write Python Scripts and the console

Tip: Find a detailed overview of the KNIME-Python Integration in KNIME Docs plus tips on how to set it up in the blog article “How to Set Up Your Python Extensions”.

Adopt Bundled Packages for Common Functionalities

The Python Script node includes a selection of Python packages to get you started right away. KNIME automatically installs a bundled environment in the background. That means, the Python Script node can be used without installing, configuring, or even knowing about environments.

To allow you to quickly start into Python scripting without the need to touch any Python settings, the shipped bundled packages include a number of popular Python functionalities to get you started. Since not everyone needs everything, the set of packages is quite limited to accommodate many scripting situations while keeping the set of packages small. You'll find a list of the packages bundled with the Python Script node in the KNIME Python Integration Guide.

Add Additional Custom Packages as Needed

If you need to adopt a Python package that is not available in the bundled environment (Fig. 1), do not despair! You can install miniconda and add the custom packages in a new environment via the Anaconda prompt. Before adopting the custom packages in the Python Script node select the new environment in Conda Environment Propagation node. This node can then automatically configure via flow variable (see docs) any Python Script nodes downstream (Fig. 2). Via this approach, you can “propagate” that environment to any new system with Anaconda or Miniconda already installed, usually a coworker laptop or KNIME Business Hub.

Tip: Check out this blog article Manage Your Python Environments with Conda and KNIME to learn more on the Conda Environment Propagation node.

2-conda-environment-propagation-node.gif
Figure 2 : Use custom packages and ship the environment containing these packages using the Conda Environment Propagation node

Get Started with the Python Script Space

On KNIME Community Hub, you will find the Python Script Space, which contains example workflows for you to quickly learn how to use the Python Script node in your workflows. This space of examples is especially for KNIME users who are keen to use Python scripts inside KNIME. We have divided these example workflows into five groups (Fig. 3).

3-python-script-space-workflows.png
Figure 3: Python Script Space and segregation of workflows into five groups

  • 01_Getting_Started: These workflows will help you understand how to add/remove various input and output ports and access the input table or objects. These workflows will also familiarize you with the newly introduced API that makes data transfer faster between KNIME and Python. Learn more on this topic consult the KNIME Python Script API Docs.

  • 02_Using_Bundled_Python_Packages: Explore these workflows that demonstrate the usage of bundled packages introduced above. We provide examples of popular packages like NumPy, pandas, scikit-learn, and also visualizations with matplotlib and seaborn.

  • 03_Using_Custom_Python_Packages: Download these workflows which show the usage of Python packages that are not available in the bundled Python environment like Networkx and Geopy. They are merely a demonstration of how many other Python packages can be used inside KNIME with the Python Script node. At last, we also provide an example of using custom-defined Python classes inside KNIME.

  • 04_Sharing_Python_Scripts_in_Components: Access workflows that demonstrate how the workflows with the Python Script node can be abstracted into Components that are reusable and reliable for sharing these functionalities with other users.

  • 05_Jupyter_Notebook: Get an example where we trigger execution of a Jupyter Notebook from a KNIME workflow by exchanging data. To learn more about this topic check the article Cut Down Implementation Time with Jupyter and KNIME.

Drag and drop the workflow of interest from this space into your KNIME Analytics Platform, and start tweaking it to fit your problem.