The low-code KNIME Analytics Platform, an open source data science tool, includes a Python integration. Coders can choose to package scripts and make them available for re-use by non-coders, for example.
With the Python integration in KNIME you can use an existing Python installation from KNIME Analytics Platform. But what if Python is not yet installed on your system?
Here is a quick step by step guide on how to install Python, get it working in KNIME, and also configure the Deep Learning extensions. For a more detailed and technical onboarding of these features please consider browsing the contents at KNIME Docs - Integrations
The KNIME Python Integration and KNIME Deep Learning Keras Integration, as well as other deep learning integrations, are widely used by the Python-KNIME open source community. They contain nodes to integrate Python scripts from the configuration dialog as well as deep learning techniques such as Keras and TensorFlow. For the purpose of this article, we assume that Python is not yet installed on your system.
Note: If you already know how to execute scripts of Python and Keras directly from KNIME and instead are interested to find out how to code new nodes and extensions in Python, read: 4 Steps for your Python Team to Develop KNIME Nodes →
Note: The new version of the Python Script node, available from KNIME Analytics Platform 4.7, supports only Python version 3. If you require Python version 2.7 inside KNIME Analytics Platform, install the KNIME Python Integration (Legacy). To read more about the new node read the article The New Python Script node with Bundled Packages.
Install the KNIME Python Integration by Drag and Drop
You can find the Python Integration on the KNIME Community Hub. Enter "Python" in the search field and filter by “Extensions”, or click straight to the KNIME Python Integration hub page. With your version of KNIME Analytics Platform open, drag the Python Integration download icon over to the Workflow Editor.
Use Bundled Packages in Python Script node
Starting with KNIME Analytics Platform 4.7 and above, the KNIME Python Integration is pre-installed with a selection of Python packages (e.g. Python libraries) as a bundled environment. This means you can use the Python Script node without needing to install, configure, or even know about environments.
The set of packages is limited to accommodate a select number of popular scripting situations while keeping the environment small. Read more about using the bundled environment in the article The New Python Script node with Bundled Packages.
Add Custom Packages in Python Script node
If you need to adopt a custom Python package that is not available in the bundled environment, follow the steps given below
Step 1. Install Anaconda to Manage your Python Environments
Install Anaconda or Miniconda from the links below and choose the version of Python you prefer. We recommend using either the:
- Anaconda Python distribution from Continuum Analytics - the full toolkit to work with thousands of open source packages and libraries, or
-
Miniconda - a minimal installer for conda, which includes only conda, Python, the packages they depend on, plus a small number of other useful packages.
Go to KNIME Preferences via “File > Preferences > KNIME > Conda'' and provide the path to where your Anaconda installation is located. Usually this should be “C:\ProgramData\Miniconda3”.
Step 2. Create New Python Environment
If you do not have a Python environment containing all the necessary dependencies for the KNIME Python Integration, there are two options to create a new environment.
Step 3. Recommended Option: Create new environment with the Anaconda prompt
Create a Python environment that contains dependencies for KNIME Python Integration directly from your Anaconda Prompt. Using the command, you can copy and paste from the “Create New Conda Environment” KNIME Docs section.
To create an environment with Python 3.10 you can change the last digits of this command.
Step 4. Alternative Option: Create new environment from KNIME Preference page
In your installation of KNIME Analytics Platform, go to the Preferences page under File menu. Under KNIME you'll find up to four Python-related options (based on the installed extensions): Python, Python (legacy), Python Deep Learning, and Python Scripting. (Python Scripting is a community extension, which we won’t cover here in this article.)
Click the Python option to open the Python preferences.
Select Conda under Python environment configuration as shown in Figure 2, or point to Python start scripts that activate a suitable environment you created manually (by selecting the Manual sub option).
Click the “New environment…” button. This opens the following dialog (Figure 3). Provide a name for the new environment and click "Create new environment". This creates a new conda environment containing all the required dependencies for the KNIME Python Integration.
Once the environment is successfully created, the dialog closes and the new environment is selected automatically. If everything worked out fine, the Python version is now shown below the environment selection and you are ready to go. After the environment is created, you can run Jupyter notebooks as well as the Python-based nodes such as Python scripting, Python Object Reader etc.
Step 5. Install Custom Packages
After the Python Environment is successfully created, open the Anaconda Prompt and type commands there to activate the environment you just created and install the custom packages that are still missing.
Step 6. Set Up the Python Environment
Once the custom packages are installed to the environment you just created, you can use these packages inside KNIME Analytics Platform by setting up the Python environment, which is shown in Figure 4.
Go to the Preferences page under File menu and select Python. Select the Conda option and choose the environment that contains packages you need. Click “Apply and Close” to finish. Now start using the custom packages inside the Python Script node and it will be executed successfully.
If you want to share or export the workflow that contains a Python Script node which uses custom packages. Consider using the Conda Environment Propagation node, it will make sure the dependent files are also transferred with the workflow. To understand this in detail, read the article Manage Your Python Environments with Conda and KNIME.
Step 7. Example workflows that use the KNIME Python Integration
The Python Script Space on KNIME Community Hub is a repository of examples for you to quickly learn how to use the Python Script node in your workflows. This space caters to KNIME users who are keen on using Python scripts inside KNIME.
Set Up your Python Deep Learning Environment
Step 1. Install a KNIME Deep Learning Integration
If you want to use Python for deep learning, you will need to install one of the KNIME Deep Learning Integrations as well. You’ll find different deep learning integrations on the KNIME Community Hub, for example for Keras, TensorFlow, TensorFlow2, ONNX, etc. Choose the one(s) that you need for your project. Follow the same procedure as above to locate them on the Hub and install them on your system.
Step 2. Set Up the Deep Learning Environment
In your installation of KNIME Analytics Platform, go to the Preferences page in the File menu. Under KNIME, click the Python Deep Learning option. In the Python Deep Learning preferences (Figure 5) you have two options for configuring the Python Deep Learning environment.
Step 3. Recommended Option: “Use special Deep Learning configuration”
If you select this option, KNIME Deep Learning will use a different Python 3 environment to the one configured in the Python Preference page.
Similar to the Python preferences page, you can either automatically create Python 3 environments containing all required packages (by selecting the Conda sub option), or point to Python start scripts that activate a suitable environment you created manually (by selecting the Manual sub option).
If you have already set up a Python Deep Learning environment containing all the necessary dependencies for KNIME Deep Learning, just select it from the list and you are ready to go. If you do not have a suitable environment available, click the “New environment…” button. This opens the following dialog (Figure 5).
Step 4. Advanced Option: “Use configuration from the Python preference page”
If you choose this option, KNIME Deep Learning will use the same Python 3 environment as configured in the Python preference page. This assumes that this Python 3 environment already contains all required Python Deep Learning packages (which have to be installed manually).
A list of the required Python Deep Learning packages for the Keras Integration is documented in the KNIME Keras Integration Installation section. This option is intended for advanced users, we recommend choosing Option 1.
Using Together: Python Script and Deep Learning nodes
If you want to use Python Script nodes next to nodes from deep learning extensions (e.g. Keras Learner), you can choose to use either the same environment or a different one.
- When using different environments you can use the current Python Script node using the settings discussed in the previous section “Using Custom Packages in Python Script node”.
- When using the same deep learning environment for both nodes type you can use the Python Script (Legacy) node using the settings discussed in the previous section “Set Up your Python Deep Learning Environment”.
In either case you can control which environment the Python Script node is executing via a Conda Environment node. Learn more about this feature in the article Manage Your Python Environments with Conda and KNIME.
Python-Related Resources at a Glance
This article was designed as a quick guide to get you started with your Python integrations. Below you can find more Python-KNIME related content: workflows, blogs/articles and documentation. Happy Py-KNIMEing!