Creating Nodes for Node-RED Part 1 (python-venv)
Introduction
I created a node for Node-RED that can use the Python virtual environment.
I have created a node for Node-RED MCU before, but this is the second one for Node-RED.
I had created another node for VOICEVOX CORE first, but it has been put off because of the difficulty in validating the process of building the environment. That node also uses Python, so I will also apply the python-venv node mechanism to that node.
▼Please see GitHub for the latest program.
https://github.com/404background/node-red-contrib-python-venv
▼It is also published in npm and can be installed from Manage Palette.
https://www.npmjs.com/package/@background404/node-red-contrib-python-venv
This article is based on version 0.0.2.
Please note that future updates may change the contents.
I confirmed that it works in Windows, Linux, and Ubuntu environments now.
▼Previous articles are here
Differences in the directory structure of venv depending on the OS
It is written in the documentation, the file structure differs depending on whether you use Windows environment or not.
▼It is written in this page.
https://docs.python.org/3/library/venv.html
▼Quoting from the article, it is written as follows.
It also creates a bin (or Scripts on Windows) subdirectory containing a copy/symlink of the Python binary/binaries (as appropriate for the platform or arguments used at environment creation time).
It also creates an (initially empty) lib/pythonX.Y/site-packages subdirectory (on Windows, this is Lib\site-packages).
https://docs.python.org/3/library/venv.html
The two folder names differ as follows.
- bin (Scripts in Windows)
- lib/pythonX.Y/site-packages (Lib\site-packages in Windows)
bin/Scripts contains pip and python (.exe files in Windows). Also, site-packages contains packages installed in the virtual environment.
Execute with absolute paths
I created a node once to execute it with a relative path, but it sometimes behaved as if it executed in the directory when Node-RED was started.
To make it clear which directory to run, I made it run with an absolute path. For each of JavaScript and Python, I referred to the official reference and other pages.
For JavaScript
Use __dirname and path.dirname to get the current directory.
▼For more information about __dirname
https://nodejs.org/docs/latest/api/modules.html#__dirname
▼For more information about path.dirname(path)
https://nodejs.org/docs/latest/api/path.html#pathdirnamepath
For Python
Use__file__ to get the path of the running file and os.path.abspath(path) to get the absolute path.
▼For __file__, this article was easy to understand.
https://note.nkmk.me/python-script-file-path/
▼For more information about os.path.abspath(path)
https://docs.python.org/ja/3/library/os.path.html
Creating nodes
How nodes work
When the node is installed, setup.py is run; a command is run by preinstall, in the scripts of package.json.
▼package.json is here
{
"name": "@background404/node-red-contrib-python-venv",
"version": "0.0.1",
"description": "Node for python virtual environment",
"main": "node/venv.js",
"scripts": {
"test": "mocha test/**/*_spec.js",
"preinstall": "python setup.py"
},
"engines": {
"node": ">=14.0.0"
},
"node-red": {
"version": ">=1.3.7",
"nodes": {
"venv": "node/venv.js",
"pip": "node/pip.js"
}
},
"repository": {
"type": "git",
"url": "git+https://github.com/404background/node-red-contrib-python-venv.git"
},
"keywords": [
"node-red",
"python"
],
"author": "background404",
"license": "MIT",
"bugs": {
"url": "https://github.com/404background/node-red-contrib-python-venv/issues"
},
"homepage": "https://github.com/404background/node-red-contrib-python-venv#readme"
}
In setup.py, the Python virtual environment is created and the paths to python and pip are saved in the path.json file.
At this time, the directory name is changed depending on whether it is Windows or not.
▼setup.py is here.
import subprocess
import os
import json
absDir = os.path.dirname(os.path.abspath(__file__))
subprocess.run(['python', '-m', 'venv', 'pyenv'])
if os.name == 'nt':
subprocess.run([f'{absDir}/pyenv/Scripts/python.exe', '-m', 'pip', 'install', '--upgrade', 'pip'])
path = {
'NODE_PYENV_PYTHON': f'{absDir}/pyenv/Scripts/python.exe',
'NODE_PYENV_PIP': f'{absDir}/pyenv/Scripts/pip.exe'
}
else:
subprocess.run([f'{absDir}/pyenv/bin/python', '-m', 'pip', 'install', '--upgrade', 'pip'])
path = {
'NODE_PYENV_PYTHON': f'{absDir}/pyenv/bin/python',
'NODE_PYENV_PIP': f'{absDir}/pyenv/bin/pip'
}
with open(f'{absDir}/path.json', 'w') as f:
json.dump(path, f, indent=2)
The path written in path.json is also read by the JavaScript programs in the node.
▼The json file is used to exchange data between Python and JavaScript.
![](https://404background.com/wp-content/uploads/2024/04/image-18.png)
Structure of the venv node
In the venv node, execSync of child_process is used to execute a command that executes a Python program in a synchronous process.
By calling the python file of the virtual environment written in path.json, the process is separated from the system Python.
Normally, when using a virtual environment, you activate it before executing. In this case, the node executes it with a command, so it is called directly.
Also, config.code is blank and when the program receives msg.code, it executes it.
▼venv.js is here.
module.exports = function(RED) {
function Venv(config) {
RED.nodes.createNode(this,config)
let node = this
const fs = require('fs')
const path = require('path')
const filePath = path.join(path.dirname(__dirname), 'tmp', this.id + '.py')
const jsonPath = path.join(path.dirname(__dirname), 'path.json')
const json = fs.readFileSync(jsonPath)
const pythonPath = JSON.parse(json).NODE_PYENV_PYTHON
let code = ""
node.on('input', function(msg) {
const command = pythonPath + ' ' + filePath
if(config.code !== null && config.code !== "") {
code = config.code
} else {
code = msg.code
}
fs.writeFileSync(filePath, code)
let execSync = require('child_process').execSync
msg.payload = String(execSync(command))
node.send(msg)
})
}
RED.nodes.registerType("venv", Venv)
}
▼venv.html is here.
<script type="text/javascript">
RED.nodes.registerType('venv',{
category: 'python',
color: '#3e7cad',
defaults: {
name: {value:""},
code: {value:""}
},
inputs: 1,
outputs: 1,
icon: "function.svg",
label: function() {
return this.name||"venv";
}
});
</script>
<script type="text/html" data-template-name="venv">
<div class="form-row">
<label for="node-input-name"><i class="fa fa-tag"></i> Name</label>
<input type="text" id="node-input-name" placeholder="Name"><br>
<label for="node-input-code"><i class="fa fa-code"></i> Code:</label><br>
<textarea id="node-input-code" class="node-text-editor" style="height: 500px; min-height:300px; min-width: 100%;" placeholder="Your code here"></textarea><br>
</div>
</script>
<script type="text/html" data-help-name="venv">
</script>
▼The edit dialog looks like this.
![](https://404background.com/wp-content/uploads/2024/04/venv-node.png)
Structure of the pip node
The pip node looks exactly like the venv node, which also executes with execSync.
Commands that are frequently used with pip can be selected from a pull-down menu.
▼pip.js is here.
module.exports = function(RED) {
function Pip(config) {
RED.nodes.createNode(this,config)
let node = this
let argument = ''
let action = ''
let command = ''
const path = require('path')
const fs = require('fs')
const jsonPath = path.join(path.dirname(__dirname), 'path.json')
const json = fs.readFileSync(jsonPath)
const pathPip = JSON.parse(json).NODE_PYENV_PIP
const execSync = require('child_process').execSync
node.on('input', function(msg) {
if(config.arg !== null && config.arg !== '') {
argument = config.arg
} else {
argument = msg.payload
}
switch(config.action) {
case 'install':
action = 'install'
option = ''
break
case 'uninstall':
action = 'uninstall'
option = '-y'
break
case 'list':
action = 'list'
option = ''
argument = ''
break
default:
action = ''
option = ''
break
}
command = pathPip + ' ' + action + ' ' + option + ' ' + argument
msg.payload = String(execSync(command))
node.send(msg)
})
}
RED.nodes.registerType('pip',Pip)
}
▼pip.html is here.
<script type="text/javascript">
RED.nodes.registerType('pip',{
category: 'python',
color: '#3e7cad',
defaults: {
name: {value:""},
arg: {value:""},
action: {value:""}
},
inputs: 1,
outputs: 1,
icon: "cog.svg",
label: function() {
return this.name||"pip";
}
});
</script>
<script type="text/html" data-template-name="pip">
<div class="form-row">
<label for="node-input-name"><i class="fa fa-tag"></i> Name</label>
<input type="text" id="node-input-name" placeholder="Name"><br>
<label for="node-input-action"><i class="fa fa-cog"></i> Command</label>
<select id="node-input-action" name="action">
<option value="None">None</option>
<option value="install">install</option>
<option value="uninstall">uninstall</option>
<option value="list">list</option>
</select><br>
<label for="node-input-arg"><i class="fa fa-tag"></i> arg</label>
<input type="text" id="node-input-arg" placeholder="argument">
</div>
</script>
<script type="text/html" data-help-name="pip">
</script>
▼The edit dialog looks like this.
![](https://404background.com/wp-content/uploads/2024/04/pip-node.png)
Executing nodes
I installed and executed the nodes.
It can also be used with other existing nodes. I used it with the http in/out node.
When the node receives HTTP requests, it sends the contents of the https://example.com.
▼You can use nodes like this.
![](https://404background.com/wp-content/uploads/2024/04/sample-flow.jpg)
▼Here is a sample flow.
[{"id":"c45c04db2987653c","type":"debug","z":"3a0260d17167ea2b","name":"debug 1","active":true,"tosidebar":true,"console":false,"tostatus":false,"complete":"payload","targetType":"msg","statusVal":"","statusType":"auto","_mcu":{"mcu":false},"x":680,"y":460,"wires":[]},{"id":"a188371841023711","type":"venv","z":"3a0260d17167ea2b","name":"requests","code":"import requests\nurl = 'https://example.com/'\n \ntry:\n response = requests.get(url)\n\n if response.status_code == 200:\n print(response.text)\n else:\n print(f\"Failed to retrieve data. Status code: {response.status_code}\")\nexcept requests.RequestException as e:\n print(f\"An error occurred: {e}\")","_mcu":{"mcu":false},"x":440,"y":460,"wires":[["c45c04db2987653c","2e1f090ebb65b6fe"]]},{"id":"7ba222816f58207f","type":"pip","z":"3a0260d17167ea2b","name":"pip install requests","arg":"requests","action":"install","_mcu":{"mcu":false},"x":470,"y":240,"wires":[["49c3fb2e3c89c0c5"]]},{"id":"49c3fb2e3c89c0c5","type":"debug","z":"3a0260d17167ea2b","name":"debug 4","active":true,"tosidebar":true,"console":false,"tostatus":false,"complete":"false","statusVal":"","statusType":"auto","_mcu":{"mcu":false},"x":680,"y":240,"wires":[]},{"id":"e8f445ef70a07477","type":"inject","z":"3a0260d17167ea2b","name":"","props":[{"p":"payload"},{"p":"topic","vt":"str"}],"repeat":"","crontab":"","once":false,"onceDelay":0.1,"topic":"","payload":"","payloadType":"date","_mcu":{"mcu":false},"x":260,"y":240,"wires":[["7ba222816f58207f"]]},{"id":"6804be6580cdea18","type":"inject","z":"3a0260d17167ea2b","name":"","props":[{"p":"payload"},{"p":"topic","vt":"str"}],"repeat":"","crontab":"","once":false,"onceDelay":0.1,"topic":"","payload":"","payloadType":"date","_mcu":{"mcu":false},"x":260,"y":460,"wires":[["a188371841023711"]]},{"id":"28586b0a18f1874e","type":"pip","z":"3a0260d17167ea2b","name":"pip uninstall requests","arg":"requests","action":"uninstall","_mcu":{"mcu":false},"x":480,"y":280,"wires":[["349fd43a1ec03de1"]]},{"id":"349fd43a1ec03de1","type":"debug","z":"3a0260d17167ea2b","name":"debug 5","active":true,"tosidebar":true,"console":false,"tostatus":false,"complete":"false","statusVal":"","statusType":"auto","_mcu":{"mcu":false},"x":680,"y":280,"wires":[]},{"id":"1629786d40d7cf8a","type":"inject","z":"3a0260d17167ea2b","name":"","props":[{"p":"payload"},{"p":"topic","vt":"str"}],"repeat":"","crontab":"","once":false,"onceDelay":0.1,"topic":"","payload":"","payloadType":"date","_mcu":{"mcu":false},"x":260,"y":280,"wires":[["28586b0a18f1874e"]]},{"id":"9d46cdda3e5f8f79","type":"pip","z":"3a0260d17167ea2b","name":"pip list","arg":"requests","action":"list","_mcu":{"mcu":false},"x":430,"y":580,"wires":[["800d8cecf096bea3"]]},{"id":"800d8cecf096bea3","type":"debug","z":"3a0260d17167ea2b","name":"debug 6","active":true,"tosidebar":true,"console":false,"tostatus":false,"complete":"false","statusVal":"","statusType":"auto","_mcu":{"mcu":false},"x":600,"y":580,"wires":[]},{"id":"39785fb1845c8265","type":"debug","z":"3a0260d17167ea2b","name":"debug 7","active":true,"tosidebar":true,"console":false,"tostatus":false,"complete":"payload","targetType":"msg","statusVal":"","statusType":"auto","_mcu":{"mcu":false},"x":680,"y":360,"wires":[]},{"id":"18745a928deae57c","type":"venv","z":"3a0260d17167ea2b","name":"hello world","code":"print('hello, world!')","_mcu":{"mcu":false},"x":450,"y":360,"wires":[["39785fb1845c8265"]]},{"id":"bc8d4257965e746c","type":"inject","z":"3a0260d17167ea2b","name":"","props":[{"p":"payload"},{"p":"topic","vt":"str"}],"repeat":"","crontab":"","once":false,"onceDelay":0.1,"topic":"","payload":"","payloadType":"date","_mcu":{"mcu":false},"x":260,"y":360,"wires":[["18745a928deae57c"]]},{"id":"24ce53db449abb87","type":"http in","z":"3a0260d17167ea2b","name":"","url":"/url","method":"get","upload":false,"swaggerDoc":"","_mcu":{"mcu":false},"x":270,"y":500,"wires":[["a188371841023711"]]},{"id":"2e1f090ebb65b6fe","type":"http response","z":"3a0260d17167ea2b","name":"","statusCode":"","headers":{},"_mcu":{"mcu":false},"x":670,"y":500,"wires":[]},{"id":"a171026a86c55d5c","type":"exec","z":"3a0260d17167ea2b","command":"pip list","addpay":"","append":"","useSpawn":"false","timer":"","winHide":false,"oldrc":false,"name":"","_mcu":{"mcu":false},"x":430,"y":660,"wires":[["2aa6fd50aad3dba9"],[],["94273bb3a668778f"]]},{"id":"dea9652985cd396c","type":"inject","z":"3a0260d17167ea2b","name":"","props":[{"p":"payload"},{"p":"topic","vt":"str"}],"repeat":"","crontab":"","once":false,"onceDelay":0.1,"topic":"","payload":"","payloadType":"date","_mcu":{"mcu":false},"x":260,"y":660,"wires":[["a171026a86c55d5c"]]},{"id":"2aa6fd50aad3dba9","type":"debug","z":"3a0260d17167ea2b","name":"debug 8","active":true,"tosidebar":true,"console":false,"tostatus":false,"complete":"false","statusVal":"","statusType":"auto","_mcu":{"mcu":false},"x":600,"y":640,"wires":[]},{"id":"94273bb3a668778f","type":"debug","z":"3a0260d17167ea2b","name":"debug 9","active":true,"tosidebar":true,"console":false,"tostatus":false,"complete":"false","statusVal":"","statusType":"auto","_mcu":{"mcu":false},"x":600,"y":680,"wires":[]},{"id":"57fcebe823aa5c6f","type":"inject","z":"3a0260d17167ea2b","name":"","props":[{"p":"payload"},{"p":"topic","vt":"str"}],"repeat":"","crontab":"","once":false,"onceDelay":0.1,"topic":"","payload":"","payloadType":"date","_mcu":{"mcu":false},"x":260,"y":580,"wires":[["9d46cdda3e5f8f79"]]}]
I could also separate the packages with pip in the system and pip in the pip node.
▼The view is different from the pip list command executed in the exec node.
![](https://404background.com/wp-content/uploads/2024/04/pip-list.jpg)
Finally
Although Node-RED is JavaScript-based, I created a node that can run programs in a Python virtual environment inside it.
The flow of the program could be represented as a flow in Node-RED, and internally executed in Python.
I think I have created it in a simple form. However, I am not particularly knowledgeable in either JavaScript or Python, so if you have any suggestions or fixes, I would appreciate it if you could send them to me in an issue or comment.
Discover more from 趣味的ロボット研究所
Subscribe to get the latest posts sent to your email.