Creating Nodes for Node-RED Part 1 (python-venv)

2024年7月10日 Last updated : 2024年7月10日 管理人

Introduction

　I created a node for Node-RED that can use the Python virtual environment.

　I have created a node for Node-RED MCU before, but this is the second one for Node-RED.

　I had created another node for VOICEVOX CORE first, but it has been put off because of the difficulty in validating the process of building the environment. That node also uses Python, so I will also apply the python-venv node mechanism to that node.

▼Please see GitHub for the latest program.

https://github.com/404background/node-red-contrib-python-venv

▼It is also published in npm and can be installed from Manage Palette.

https://www.npmjs.com/package/@background404/node-red-contrib-python-venv

This article is based on version 0.0.2.
Please note that future updates may change the contents.
I confirmed that it works in Windows, Linux, and Ubuntu environments now.

▼Previous articles are here

Node-RED MCU用のノードを作成してみる　その3（Serialノード）

はじめに 　今回はNode-RED MCU用のSerialノードを作成してみました。デバッガツールとは元々シリアル通信を行っているのですが、他のポートでのシリアル通信を行うことが…

趣味的ロボット研究所
Publish Created Nodes for Node-RED

Introduction 　I created a node for Node-RED MCU in a past article. In this article, I have published that node […]

趣味的ロボット研究所

Differences in the directory structure of venv depending on the OS

　It is written in the documentation, the file structure differs depending on whether you use Windows environment or not.

▼It is written in this page.

https://docs.python.org/3/library/venv.html

▼Quoting from the article, it is written as follows.

It also creates a bin (or Scripts on Windows) subdirectory containing a copy/symlink of the Python binary/binaries (as appropriate for the platform or arguments used at environment creation time). 

It also creates an (initially empty) lib/pythonX.Y/site-packages subdirectory (on Windows, this is Lib\site-packages).

https://docs.python.org/3/library/venv.html

　The two folder names differ as follows.

  • bin (Scripts in Windows)
  • lib/pythonX.Y/site-packages (Lib\site-packages in Windows)

　bin/Scripts contains pip and python (.exe files in Windows). Also, site-packages contains packages installed in the virtual environment.

Execute with absolute paths

　I created a node once to execute it with a relative path, but it sometimes behaved as if it executed in the directory when Node-RED was started.

　To make it clear which directory to run, I made it run with an absolute path. For each of JavaScript and Python, I referred to the official reference and other pages.

For JavaScript

　Use __dirname and path.dirname to get the current directory.

▼For more information about __dirname

https://nodejs.org/docs/latest/api/modules.html#__dirname

▼For more information about path.dirname(path)

https://nodejs.org/docs/latest/api/path.html#pathdirnamepath

For Python

　Use__file__ to get the path of the running file and os.path.abspath(path) to get the absolute path.

▼For __file__, this article was easy to understand.

https://note.nkmk.me/python-script-file-path/

▼For more information about os.path.abspath(path)

https://docs.python.org/ja/3/library/os.path.html

Creating nodes

How nodes work

　When the node is installed, setup.py is run; a command is run by preinstall, in the scripts of package.json.

▼package.json is here

{
  "name": "@background404/node-red-contrib-python-venv",
  "version": "0.0.1",
  "description": "Node for python virtual environment",
  "main": "node/venv.js",
  "scripts": {
    "test": "mocha test/**/*_spec.js",
    "preinstall": "python setup.py"
  },
  "engines": {
    "node": ">=14.0.0"
  },
  "node-red": {
    "version": ">=1.3.7",
    "nodes": {
      "venv": "node/venv.js",
      "pip": "node/pip.js"
    }
  },
  "repository": {
    "type": "git",
    "url": "git+https://github.com/404background/node-red-contrib-python-venv.git"
  },
  "keywords": [
    "node-red",
    "python"
  ],
  "author": "background404",
  "license": "MIT",
  "bugs": {
    "url": "https://github.com/404background/node-red-contrib-python-venv/issues"
  },
  "homepage": "https://github.com/404background/node-red-contrib-python-venv#readme"
}

　In setup.py, the Python virtual environment is created and the paths to python and pip are saved in the path.json file.

　At this time, the directory name is changed depending on whether it is Windows or not.

▼setup.py is here.

import subprocess
import os
import json

absDir = os.path.dirname(os.path.abspath(__file__))
subprocess.run(['python', '-m', 'venv', 'pyenv'])

if os.name == 'nt':
    subprocess.run([f'{absDir}/pyenv/Scripts/python.exe', '-m', 'pip', 'install', '--upgrade', 'pip'])
    path = {
        'NODE_PYENV_PYTHON': f'{absDir}/pyenv/Scripts/python.exe',
        'NODE_PYENV_PIP': f'{absDir}/pyenv/Scripts/pip.exe'
    }
else:
    subprocess.run([f'{absDir}/pyenv/bin/python', '-m', 'pip', 'install', '--upgrade', 'pip'])
    path = {
        'NODE_PYENV_PYTHON': f'{absDir}/pyenv/bin/python',
        'NODE_PYENV_PIP': f'{absDir}/pyenv/bin/pip'
    }

with open(f'{absDir}/path.json', 'w') as f:
    json.dump(path, f, indent=2)

　The path written in path.json is also read by the JavaScript programs in the node.

▼The json file is used to exchange data between Python and JavaScript.

Structure of the venv node

　In the venv node, execSync of child_process is used to execute a command that executes a Python program in a synchronous process.

　By calling the python file of the virtual environment written in path.json, the process is separated from the system Python.

　Normally, when using a virtual environment, you activate it before executing. In this case, the node executes it with a command, so it is called directly.

　Also, config.code is blank and when the program receives msg.code, it executes it.

▼venv.js is here.

module.exports = function(RED) {
    function Venv(config) {
        RED.nodes.createNode(this,config)
        let node = this

        const fs = require('fs')
        const path = require('path')
        const filePath = path.join(path.dirname(__dirname), 'tmp', this.id + '.py')
        const jsonPath = path.join(path.dirname(__dirname), 'path.json')
        const json = fs.readFileSync(jsonPath)
        const pythonPath = JSON.parse(json).NODE_PYENV_PYTHON
        let code = ""

        node.on('input', function(msg) {
            const command = pythonPath + ' ' + filePath

            if(config.code !== null && config.code !== "") {
                code = config.code
            } else {
                code = msg.code
            }
            fs.writeFileSync(filePath, code)
            
            let execSync = require('child_process').execSync
            msg.payload = String(execSync(command))
            node.send(msg)
        })
    }
    RED.nodes.registerType("venv", Venv)
}

▼venv.html is here.

<script type="text/javascript">
    RED.nodes.registerType('venv',{
        category: 'python',
        color: '#3e7cad',
        defaults: {
            name: {value:""},
            code: {value:""}
        },
        inputs: 1,
        outputs: 1,
        icon: "function.svg",
        label: function() {
            return this.name||"venv";
        }
    });
</script>

<script type="text/html" data-template-name="venv">
    <div class="form-row">
        <label for="node-input-name"><i class="fa fa-tag"></i> Name</label>
        <input type="text" id="node-input-name" placeholder="Name"><br>
        <label for="node-input-code"><i class="fa fa-code"></i> Code:</label><br>
        <textarea id="node-input-code" class="node-text-editor" style="height: 500px; min-height:300px; min-width: 100%;" placeholder="Your code here"></textarea><br>
    </div>
</script>

<script type="text/html" data-help-name="venv">

</script>

▼The edit dialog looks like this.

Structure of the pip node

　The pip node looks exactly like the venv node, which also executes with execSync.

　Commands that are frequently used with pip can be selected from a pull-down menu.

▼pip.js is here.

module.exports = function(RED) {
    function Pip(config) {
        RED.nodes.createNode(this,config)
        let node = this
        let argument = ''
        let action = ''
        let command = ''
        const path = require('path')
        const fs = require('fs')
        const jsonPath = path.join(path.dirname(__dirname), 'path.json')
        const json = fs.readFileSync(jsonPath)
        const pathPip = JSON.parse(json).NODE_PYENV_PIP
        const execSync = require('child_process').execSync

        node.on('input', function(msg) {
            if(config.arg !== null && config.arg !== '') {
                argument = config.arg
            } else {
                argument = msg.payload
            }

            switch(config.action) {
                case 'install':
                    action = 'install'
                    option =  ''
                    break
                case 'uninstall':
                    action = 'uninstall'
                    option =  '-y'
                    break
                case 'list':
                    action = 'list'
                    option =  ''
                    argument = ''
                    break
                default:
                    action = ''
                    option = ''
                    break
            }
            command = pathPip + ' ' + action + ' ' + option + ' ' + argument
            msg.payload = String(execSync(command))
            node.send(msg)
        })
    }
    RED.nodes.registerType('pip',Pip)
}

▼pip.html is here.

<script type="text/javascript">
    RED.nodes.registerType('pip',{
        category: 'python',
        color: '#3e7cad',
        defaults: {
            name: {value:""},
            arg: {value:""},
            action: {value:""}
        },
        inputs: 1,
        outputs: 1,
        icon: "cog.svg",
        label: function() {
            return this.name||"pip";
        }
    });
</script>

<script type="text/html" data-template-name="pip">
    <div class="form-row">
        <label for="node-input-name"><i class="fa fa-tag"></i> Name</label>
        <input type="text" id="node-input-name" placeholder="Name"><br>
        <label for="node-input-action"><i class="fa fa-cog"></i> Command</label>
        <select id="node-input-action" name="action">
            <option value="None">None</option>
            <option value="install">install</option>
            <option value="uninstall">uninstall</option>
            <option value="list">list</option>
        </select><br>
        <label for="node-input-arg"><i class="fa fa-tag"></i> arg</label>
        <input type="text" id="node-input-arg" placeholder="argument">
    </div>
</script>

<script type="text/html" data-help-name="pip">

</script>

▼The edit dialog looks like this.

Executing nodes

　I installed and executed the nodes.

　It can also be used with other existing nodes. I used it with the http in/out node.

　When the node receives HTTP requests, it sends the contents of the https://example.com.

▼You can use nodes like this.

▼Here is a sample flow.

[{"id":"c45c04db2987653c","type":"debug","z":"3a0260d17167ea2b","name":"debug 1","active":true,"tosidebar":true,"console":false,"tostatus":false,"complete":"payload","targetType":"msg","statusVal":"","statusType":"auto","_mcu":{"mcu":false},"x":680,"y":460,"wires":[]},{"id":"a188371841023711","type":"venv","z":"3a0260d17167ea2b","name":"requests","code":"import requests\nurl = 'https://example.com/'\n    \ntry:\n    response = requests.get(url)\n\n    if response.status_code == 200:\n        print(response.text)\n    else:\n        print(f\"Failed to retrieve data. Status code: {response.status_code}\")\nexcept requests.RequestException as e:\n    print(f\"An error occurred: {e}\")","_mcu":{"mcu":false},"x":440,"y":460,"wires":[["c45c04db2987653c","2e1f090ebb65b6fe"]]},{"id":"7ba222816f58207f","type":"pip","z":"3a0260d17167ea2b","name":"pip install requests","arg":"requests","action":"install","_mcu":{"mcu":false},"x":470,"y":240,"wires":[["49c3fb2e3c89c0c5"]]},{"id":"49c3fb2e3c89c0c5","type":"debug","z":"3a0260d17167ea2b","name":"debug 4","active":true,"tosidebar":true,"console":false,"tostatus":false,"complete":"false","statusVal":"","statusType":"auto","_mcu":{"mcu":false},"x":680,"y":240,"wires":[]},{"id":"e8f445ef70a07477","type":"inject","z":"3a0260d17167ea2b","name":"","props":[{"p":"payload"},{"p":"topic","vt":"str"}],"repeat":"","crontab":"","once":false,"onceDelay":0.1,"topic":"","payload":"","payloadType":"date","_mcu":{"mcu":false},"x":260,"y":240,"wires":[["7ba222816f58207f"]]},{"id":"6804be6580cdea18","type":"inject","z":"3a0260d17167ea2b","name":"","props":[{"p":"payload"},{"p":"topic","vt":"str"}],"repeat":"","crontab":"","once":false,"onceDelay":0.1,"topic":"","payload":"","payloadType":"date","_mcu":{"mcu":false},"x":260,"y":460,"wires":[["a188371841023711"]]},{"id":"28586b0a18f1874e","type":"pip","z":"3a0260d17167ea2b","name":"pip uninstall requests","arg":"requests","action":"uninstall","_mcu":{"mcu":false},"x":480,"y":280,"wires":[["349fd43a1ec03de1"]]},{"id":"349fd43a1ec03de1","type":"debug","z":"3a0260d17167ea2b","name":"debug 5","active":true,"tosidebar":true,"console":false,"tostatus":false,"complete":"false","statusVal":"","statusType":"auto","_mcu":{"mcu":false},"x":680,"y":280,"wires":[]},{"id":"1629786d40d7cf8a","type":"inject","z":"3a0260d17167ea2b","name":"","props":[{"p":"payload"},{"p":"topic","vt":"str"}],"repeat":"","crontab":"","once":false,"onceDelay":0.1,"topic":"","payload":"","payloadType":"date","_mcu":{"mcu":false},"x":260,"y":280,"wires":[["28586b0a18f1874e"]]},{"id":"9d46cdda3e5f8f79","type":"pip","z":"3a0260d17167ea2b","name":"pip list","arg":"requests","action":"list","_mcu":{"mcu":false},"x":430,"y":580,"wires":[["800d8cecf096bea3"]]},{"id":"800d8cecf096bea3","type":"debug","z":"3a0260d17167ea2b","name":"debug 6","active":true,"tosidebar":true,"console":false,"tostatus":false,"complete":"false","statusVal":"","statusType":"auto","_mcu":{"mcu":false},"x":600,"y":580,"wires":[]},{"id":"39785fb1845c8265","type":"debug","z":"3a0260d17167ea2b","name":"debug 7","active":true,"tosidebar":true,"console":false,"tostatus":false,"complete":"payload","targetType":"msg","statusVal":"","statusType":"auto","_mcu":{"mcu":false},"x":680,"y":360,"wires":[]},{"id":"18745a928deae57c","type":"venv","z":"3a0260d17167ea2b","name":"hello world","code":"print('hello, world!')","_mcu":{"mcu":false},"x":450,"y":360,"wires":[["39785fb1845c8265"]]},{"id":"bc8d4257965e746c","type":"inject","z":"3a0260d17167ea2b","name":"","props":[{"p":"payload"},{"p":"topic","vt":"str"}],"repeat":"","crontab":"","once":false,"onceDelay":0.1,"topic":"","payload":"","payloadType":"date","_mcu":{"mcu":false},"x":260,"y":360,"wires":[["18745a928deae57c"]]},{"id":"24ce53db449abb87","type":"http in","z":"3a0260d17167ea2b","name":"","url":"/url","method":"get","upload":false,"swaggerDoc":"","_mcu":{"mcu":false},"x":270,"y":500,"wires":[["a188371841023711"]]},{"id":"2e1f090ebb65b6fe","type":"http response","z":"3a0260d17167ea2b","name":"","statusCode":"","headers":{},"_mcu":{"mcu":false},"x":670,"y":500,"wires":[]},{"id":"a171026a86c55d5c","type":"exec","z":"3a0260d17167ea2b","command":"pip list","addpay":"","append":"","useSpawn":"false","timer":"","winHide":false,"oldrc":false,"name":"","_mcu":{"mcu":false},"x":430,"y":660,"wires":[["2aa6fd50aad3dba9"],[],["94273bb3a668778f"]]},{"id":"dea9652985cd396c","type":"inject","z":"3a0260d17167ea2b","name":"","props":[{"p":"payload"},{"p":"topic","vt":"str"}],"repeat":"","crontab":"","once":false,"onceDelay":0.1,"topic":"","payload":"","payloadType":"date","_mcu":{"mcu":false},"x":260,"y":660,"wires":[["a171026a86c55d5c"]]},{"id":"2aa6fd50aad3dba9","type":"debug","z":"3a0260d17167ea2b","name":"debug 8","active":true,"tosidebar":true,"console":false,"tostatus":false,"complete":"false","statusVal":"","statusType":"auto","_mcu":{"mcu":false},"x":600,"y":640,"wires":[]},{"id":"94273bb3a668778f","type":"debug","z":"3a0260d17167ea2b","name":"debug 9","active":true,"tosidebar":true,"console":false,"tostatus":false,"complete":"false","statusVal":"","statusType":"auto","_mcu":{"mcu":false},"x":600,"y":680,"wires":[]},{"id":"57fcebe823aa5c6f","type":"inject","z":"3a0260d17167ea2b","name":"","props":[{"p":"payload"},{"p":"topic","vt":"str"}],"repeat":"","crontab":"","once":false,"onceDelay":0.1,"topic":"","payload":"","payloadType":"date","_mcu":{"mcu":false},"x":260,"y":580,"wires":[["9d46cdda3e5f8f79"]]}]

　I could also separate the packages with pip in the system and pip in the pip node.

▼The view is different from the pip list command executed in the exec node.

Finally

　Although Node-RED is JavaScript-based, I created a node that can run programs in a Python virtual environment inside it.

　The flow of the program could be represented as a flow in Node-RED, and internally executed in Python.

　I think I have created it in a simple form. However, I am not particularly knowledgeable in either JavaScript or Python, so if you have any suggestions or fixes, I would appreciate it if you could send them to me in an issue or comment.

Categories
Programming
Tags

Leave a Reply

Your email address will not be published. Required fields are marked *

Next article
Control Motors with Motor Driver Part 1 (TB6612FNG)
2024年7月12日