Skip to content

LNROOM #14 - Understand CLN Plugin mechanism with a Bash example

In this live, we add the method mymethod to Core Lightning by writing a dynamic Bash plugin called myplugin.bash. Doing this, we try to understand: (1) how startup options are passed to the plugin, (2) how cli parameters are passed to the plugin and (3) how to communicate to the lightning node via JSON-RPC over unix sockets.

Transcript with corrections and improvements

In this episode, we write a plugin in Bash that registers a JSON-RPC method to lightningd called mymethod.

Assuming l1-cli is an alias for lightning-cli --lightning-dir=/tmp/l1-regtest, when we start our plugin myplugin.bash with the startup option foo_opt set to BAR like this

â—‰ tony@tony:~/lnroom:
$ l1-cli -k plugin subcommand=start plugin=$(pwd)/myplugin.bash foo_opt=BAR

we want mymethod method to return

  1. the node id of the node running the plugin in the id field,
  2. the startup options in the options field and
  3. the cli parameters in the cli_params field.

Specifically, we want this:

â—‰ tony@tony:~/lnroom:
$ l1-cli -k mymethod foo1=bar1 foo2=bar2
{
   "id": "0204663461c108e470b83a111c241b34cb79e6661890a78a211bbfdb909c934b59",
   "options": {
      "foo_opt": "BAR"
   },
   "cli_params": {
      "foo1": "bar1",
      "foo2": "bar2"
   }
}

Core Lightning plugin system

Before implementing that plugin, let's describe Core Lightning plugin system.

Once lightningd has started the plugin as a subprocess, the plugin and lightningd communicate using pipes. Specifically,

  1. lightningd writes to the stdin stream of the plugin to send it messages and
  2. the plugin writes to its stdout stream to send messages to lightningd.

To understand each other they use the JSON-RPC protocol.

That communcation can be represented like this:

          ->  stdin
         |        |
lightningd        plugin
         |        |
          <- stdout

Now that we know how lightningd and plugins communicate with each other, let's take a look at the life cycle of a plugin like myplugin.bash which registers a JSON-RPC method called mymethod:

  1. When lightningd starts myplugin.bash, it sends the getmanifest request to myplugin.bash. myplugin.bash replies to lightningd with a response containing the declaration of the startup option foo_opt and the information to register mymethod JSON-RPC method.

  2. Then lightningd sends the init request to myplugin.bash. This tells myplugin.bash that lightningd is ready to communicate. That init request contains informations that the plugin might needs to work correctly. For instance, myplugin.bash needs to know the value of the startup option foo_opt and to know the Unix socket file to use to connect to the node and to send JSON-RPC requests in order to retrieve the node id. Those informations are available in the init request.

  3. Then myplugin.bash starts an IO loop and waits in its stdin stream incoming requests from lightningd:

    • (a). A client sends a mymethod request to lightningd. As myplugin.bash has registered mymethod JSON-RPC method to lightningd, lightningd forwards that request to myplugin.bash,
    • (b). myplugin.bash receives that request, builds a response and sends it back to lightningd,
    • ©. Finally, lightningd forwards that response to the client,
    • (d). We repeat (a) to ©.

Setup

Here is my setup

â—‰ tony@tony:~/lnroom:
$ ./setup.sh
Ubuntu 22.04.2 LTS
lightningd v23.02.2
GNU bash, version 5.1.16(1)-release (x86_64-pc-linux-gnu)

Start 2 Lightning nodes running on regtest

Let's start two Lightning nodes running on the Bitcoin regtest chain by sourcing the script lightning/contrib/startup_regtest.sh provided in CLN repository and by running the command start_ln:

â—‰ tony@tony:~/lnroom:
$ source lightning/contrib/startup_regtest.sh
lightning-cli is hashed (/usr/local/bin/lightning-cli)
lightningd is /usr/local/bin/lightningd
Useful commands:
  start_ln 3: start three nodes, l1, l2, l3
  connect 1 2: connect l1 and l2
  fund_nodes: connect all nodes with channels, in a row
  stop_ln: shutdown
  destroy_ln: remove ln directories
â—‰ tony@tony:~/lnroom:
$ start_ln
Bitcoin Core starting
awaiting bitcoind...
Making "default" bitcoind wallet.
[1] 1139027
[2] 1139063
WARNING: eatmydata not found: instal it for faster testing
Commands:
        l1-cli, l1-log,
        l2-cli, l2-log,
        bt-cli, stop_ln, fund_nodes

We can check that l1-cli is just an alias for lightning-cli with the base directory being /tmp/l1-regtest:

â—‰ tony@tony:~/lnroom:
$ alias l1-cli
alias l1-cli='lightning-cli --lightning-dir=/tmp/l1-regtest'

Write in the /tmp/myplugin file

When we start myplugin.bash script as a plugin with lightningd

#!/usr/bin/bash

set -e

we get the following error because the script stops before replying to the getmanifest request:

â—‰ tony@tony:~/lnroom:
$ l1-cli plugin start $(pwd)/myplugin.bash
{
   "code": -3,
   "message": "/home/tony/lnroom/myplugin.bash: exited before replying to getmanifest"
}

Note that myplugin.bash is an executable file.

As plugins use their stdout stream to send messages to lightningd, plugins can't write to their stdout to give us (the programmers) information about their behavior.

Indeed, if we modify myplugin.bash to be

#!/usr/bin/bash

set -e

echo foo

we get the same error as above and foo didn't get printed in our terminal:

â—‰ tony@tony:~/lnroom:
$ l1-cli plugin start $(pwd)/myplugin.bash
{
   "code": -3,
   "message": "/home/tony/lnroom/myplugin.bash: exited before replying to getmanifest"
}

Not a problem, in this video to get informations about myplugin.bash we'll write stuff in /tmp/myplugin file like this:

#!/usr/bin/bash

set -e

[[ -e /tmp/myplugin ]] && rm /tmp/myplugin

echo foo >> /tmp/myplugin

Let's try to start our plugin again:

â—‰ tony@tony:~/lnroom:
$ l1-cli plugin start $(pwd)/myplugin.bash
{
   "code": -3,
   "message": "/home/tony/lnroom/myplugin.bash: exited before replying to getmanifest"
}

We get the same error but this time we wrote foo in /tmp/myplugin:

foo

This is our tool to try things out and to get informations about the "state" of myplugin.bash plugin.

getmanifest request

Let's treat the getmanifest request.

First we receive the getmanifest request and store it in the variable JSON that we wrote in /tmp/myplugin:

...
# getmanifest

read -r JSON
read -r _
echo "$JSON" >> /tmp/myplugin

Note that the second read is to read the empty line as lightningd request ends with two consecutive new lines (\n\n).

We get the same error as before trying to start myplugin.bash plugin

â—‰ tony@tony:~/lnroom:
$ l1-cli plugin start $(pwd)/myplugin.bash
{
   "code": -3,
   "message": "/home/tony/lnroom/myplugin.bash: exited before replying to getmanifest"
}

but this time we wrote the getmanifest request in /tmp/myplugin that we can prettify like this:

{
  "jsonrpc": "2.0",
  "id": 52,
  "method": "getmanifest",
  "params": {
    "allow-deprecated-apis": false
  }
}

Let's see if we can extract the id of that request.

To do so we use jq utility and modify myplugin.bash like this:

...
# getmanifest

read -r JSON
read -r _

id=$(echo "$JSON" | jq .id)
echo "$JSON" >> /tmp/myplugin
echo "$id" >> /tmp/myplugin

We get the same error as before trying to start myplugin.bash plugin

â—‰ tony@tony:~/lnroom:
$ l1-cli plugin start $(pwd)/myplugin.bash
{
   "code": -3,
   "message": "/home/tony/lnroom/myplugin.bash: exited before replying to getmanifest"
}

but now we can check in /tmp/myplugin that we correctly extracted the request id:

{"jsonrpc":"2.0","id":59,"method":"getmanifest","params":{"allow-deprecated-apis":false}}
59

The moment to reply to the getmanifest request has come.

In the getmanifest response, we tells lightningd that:

  1. we want the plugin to be dynamic,
  2. we want to declare foo_opt startup option with bar as default value and
  3. we want to register mymethod JSON-RPC method.

To do so, we use the fields dynamic, options and rpcmethods like this (where "id" will be changed in the Bash script to the value we received from lightningd):

{
  "jsonrpc": "2.0",
  "id": "id",
  "result": {
    "dynamic": true,
    "options": [
      {
        "name": "foo_opt",
        "type": "string",
        "default": "bar",
        "description": "description"
      }
    ],
    "rpcmethods": [
      {
        "name": "mymethod",
        "usage": "",
        "description": "description"
      }
    ]
  }
}

So we modify myplugin.bash like this:

...
# getmanifest

read -r JSON
read -r _
id=$(echo "$JSON" | jq .id)
echo '{"jsonrpc": "2.0", "id": '"$id"', "result": {"dynamic": true, "options": [{"name": "foo_opt", "type": "string", "default": "bar", "description": "description"}], "rpcmethods": [{"name": "mymethod", "usage": "", "description": "description"}]}}'

Back to our terminal, we try to start our plugin and we get a new error because the plugin doesn't reply to the init request:

â—‰ tony@tony:~/lnroom:
$ l1-cli plugin start $(pwd)/myplugin.bash
{
   "code": -3,
   "message": "/home/tony/lnroom/myplugin.bash: exited before replying to init"
}

init request

Let's modify myplugin.bash such that we can write the init request and its id to /tmp/myplugin file:

...
# init

read -r JSON
read -r _

id=$(echo "$JSON" | jq .id)
echo "$JSON" >> /tmp/myplugin
echo "$id" >> /tmp/myplugin

When we try to start our plugin still get the same error

â—‰ tony@tony:~/lnroom:
$ l1-cli plugin start $(pwd)/myplugin.bash
{
   "code": -3,
   "message": "/home/tony/lnroom/myplugin.bash: exited before replying to init"
}

but now we wrote the init request and its id to /tmp/myplugin which is now

{"jsonrpc":"2.0","id":"cln:init#79","method":"init","params":{"options":{"foo_opt":"bar"},"configuration":{"lightning-dir":"/tmp/l1-regtest/regtest","rpc-file":"lightning-rpc","startup":false,"network":"regtest","feature_set":{"init":"08a000080269a2","node":"88a000080269a2","channel":"","invoice":"02000000024100"}}}}
"cln:init#79"

and we can prettify the init request like this:

{
  "jsonrpc": "2.0",
  "id": "cln:init#79",
  "method": "init",
  "params": {
    "options": {
      "foo_opt": "bar"
    },
    "configuration": {
      "lightning-dir": "/tmp/l1-regtest/regtest",
      "rpc-file": "lightning-rpc",
      "startup": false,
      "network": "regtest",
      "feature_set": {
        "init": "08a000080269a2",
        "node": "88a000080269a2",
        "channel": "",
        "invoice": "02000000024100"
      }
    }
  }
}

In the field options we have the value of foo_opt startup option. And with the value of the fields lightningd-dir and rpc-file we can form the Unix socket file we need to connect to our node in order to get the node id of the node running the plugin.

Using jq and the following jq filter

.params.options.foo_opt

we can set the variable foo_opt to the startup option value.

Let's modify myplugin.bash accordingly and also write that value to /tmp/myplugin file:

...
# init

read -r JSON
read -r _
id=$(echo "$JSON" | jq .id)
foo_opt=$(echo "$JSON" | jq .params.options.foo_opt)
echo "$foo_opt" >> /tmp/myplugin

We try to start myplugin.bash, we get the same error

â—‰ tony@tony:~/lnroom:
$ l1-cli plugin start $(pwd)/myplugin.bash
{
   "code": -3,
   "message": "/home/tony/lnroom/myplugin.bash: exited before replying to init"
}

and /tmp/myplugin file is now:

"foo"

Using jq and the following jq filter

.params.configuration."lightning-dir" + "/" + .params.configuration."rpc-file"

we can set the variable socket_path corresponding to the node's Unix socket file.

Let's modify myplugin.bash accordingly and also write that value to /tmp/myplugin file:

...
# init

read -r JSON
read -r _
id=$(echo "$JSON" | jq .id)
foo_opt=$(echo "$JSON" | jq .params.options.foo_opt)
socket_path=$(echo "$JSON" | jq '.params.configuration."lightning-dir" + "/" + .params.configuration."rpc-file"' -r)
echo "$socket_path" >> /tmp/myplugin

We try to start myplugin.bash, we get the same error

â—‰ tony@tony:~/lnroom:
$ l1-cli plugin start $(pwd)/myplugin.bash
{
   "code": -3,
   "message": "/home/tony/lnroom/myplugin.bash: exited before replying to init"
}

and /tmp/myplugin file is now:

/tmp/l1-regtest/regtest/lightning-rpc

So far, we extracted the information we need from the init request, but we didn't reply to that request.

As we do nothing special at that stage, we just return the following response (where "id" will be changed in the Bash script to the value we received from lightningd):

{
  "jsonrpc": "2.0",
  "id": "id",
  "result": {}
}

We modify myplugin.bash consequently:

...
# init

read -r JSON
read -r _
id=$(echo "$JSON" | jq .id)
foo_opt=$(echo "$JSON" | jq .params.options.foo_opt)
socket_path=$(echo "$JSON" | jq '.params.configuration."lightning-dir" + "/" + .params.configuration."rpc-file"' -r)

echo '{"jsonrpc": "2.0", "id": '"$id"', "result": {}}'

And now for the first time we start our plugin successfully:

â—‰ tony@tony:~/lnroom:
$ l1-cli plugin start $(pwd)/myplugin.bash
{
   "command": "start",
   "plugins": [
      {
         "name": "/usr/local/libexec/c-lightning/plugins/autoclean",
         "active": true,
         "dynamic": false
      },
      {
         "name": "/usr/local/libexec/c-lightning/plugins/chanbackup",
         "active": true,
         "dynamic": false
      },
      {
         "name": "/usr/local/libexec/c-lightning/plugins/bcli",
         "active": true,
         "dynamic": false
      },
      {
         "name": "/usr/local/libexec/c-lightning/plugins/commando",
         "active": true,
         "dynamic": false
      },
      {
         "name": "/usr/local/libexec/c-lightning/plugins/funder",
         "active": true,
         "dynamic": true
      },
      {
         "name": "/usr/local/libexec/c-lightning/plugins/topology",
         "active": true,
         "dynamic": false
      },
      {
         "name": "/usr/local/libexec/c-lightning/plugins/keysend",
         "active": true,
         "dynamic": false
      },
      {
         "name": "/usr/local/libexec/c-lightning/plugins/offers",
         "active": true,
         "dynamic": true
      },
      {
         "name": "/usr/local/libexec/c-lightning/plugins/pay",
         "active": true,
         "dynamic": true
      },
      {
         "name": "/usr/local/libexec/c-lightning/plugins/txprepare",
         "active": true,
         "dynamic": true
      },
      {
         "name": "/usr/local/libexec/c-lightning/plugins/spenderp",
         "active": true,
         "dynamic": false
      },
      {
         "name": "/usr/local/libexec/c-lightning/plugins/sql",
         "active": true,
         "dynamic": true
      },
      {
         "name": "/usr/local/libexec/c-lightning/plugins/bookkeeper",
         "active": true,
         "dynamic": false
      },
      {
         "name": "/home/tony/lnroom/myplugin.bash",
         "active": true,
         "dynamic": true
      }
   ]
}

In the previous output, we see /home/tony/lnroom/myplugin.bash plugin listed, but if we look for a process associated with that file we get nothing:

â—‰ tony@tony:~/lnroom:
$ ps -ax | rg myplugin
1139988 pts/1    S+     0:00 rg myplugin

This is totally normal, because after replying to the init request myplugin.bash script does nothing more and the process stops.

IO loop

Get the plugin working

Last part to take on in this video is the IO loop.

Let's start by writing to /tmp/myplugin file the request lightningd forwards us when a client calls the mymethod method. This is done like this:

...
# i/o loop

while read -r JSON; do
    read -r _
    id=$(echo "$JSON" | jq .id)
    echo "$JSON" >> /tmp/myplugin
done

Let's start myplugin.bash plugin and check that we have a process associated to our pluging:

â—‰ tony@tony:~/lnroom:
$ l1-cli plugin start $(pwd)/myplugin.bash
{
   "command": "start",
   "plugins": [...]
}
â—‰ tony@tony:~/lnroom:
$ ps -ax | rg myplugin
1140059 pts/1    S      0:00 /usr/bin/bash /home/tony/lnroom/myplugin.bash
1140078 pts/1    S+     0:00 rg myplugin

Now when we call mymethod, the terminal hangs (we can stop it with C-c key binding) because we don't answer to mymethod requests in myplugin.bash plugin:

â—‰ tony@tony:~/lnroom:
$ l1-cli mymethod
^C

Before fixing this we can take a look at mymethod which has been written in /tmp/myplugin file:

{ "jsonrpc" : "2.0", "method" : "mymethod", "id" : "cli:mymethod#1140094/cln:mymethod#130", "params" :[ ] }

Let's call mymethod with key/value arguments

â—‰ tony@tony:~/lnroom:
$ l1-cli -k mymethod foo1=bar1 foo2=bar2
^C

and observe the changes in the params field of mymethod request that are accessible in /tmp/myplugin file:

{ "jsonrpc" : "2.0", "method" : "mymethod", "id" : "cli:mymethod#1140094/cln:mymethod#130", "params" :[ ] }
{ "jsonrpc" : "2.0", "method" : "mymethod", "id" : "cli:mymethod#1140195/cln:mymethod#139", "params" :{ "foo1" : "bar1", "foo2" : "bar2"} }

Let's call mymethod with positional arguments

â—‰ tony@tony:~/lnroom:
$ l1-cli -k mymethod bar1 bar2
^C

and observe the changes in the params field of mymethod request that are accessible in /tmp/myplugin file:

{ "jsonrpc" : "2.0", "method" : "mymethod", "id" : "cli:mymethod#1140094/cln:mymethod#130", "params" :[ ] }
{ "jsonrpc" : "2.0", "method" : "mymethod", "id" : "cli:mymethod#1140195/cln:mymethod#139", "params" :{ "foo1" : "bar1", "foo2" : "bar2"} }
{ "jsonrpc" : "2.0", "method" : "mymethod", "id" : "cli:mymethod#1140214/cln:mymethod#142", "params" :[ "bar1", "bar2"] }

For the first time, we make our plugin working correctly by replying to mymethod requests with the payload being {"foo":"bar"} like this

...
# i/o loop

while read -r JSON; do
    read -r _
    id=$(echo "$JSON" | jq .id)
    echo '{"jsonrpc": "2.0", "id": '"$id"', "result": {"foo":"bar"}}'
done

We restart the plugin

â—‰ tony@tony:~/lnroom:
$ l1-cli plugin stop $(pwd)/myplugin.bash
{
   "command": "stop",
   "result": "Successfully stopped myplugin.bash."
}
â—‰ tony@tony:~/lnroom:
$ l1-cli plugin start $(pwd)/myplugin.bash
{
   "command": "start",
   "plugins": [...]
}

and we call mymethod method which returns the following:

â—‰ tony@tony:~/lnroom:
$ l1-cli mymethod
{
   "foo": "bar"
}

cli params

Now let's extract the cli parameters out of the request. To do that we modify myplugin.bash like this:

...
# i/o loop

while read -r JSON; do
    read -r _
    id=$(echo "$JSON" | jq .id)
    cli_params=$(echo "$JSON" | jq .params)
    echo '{"jsonrpc": "2.0", "id": '"$id"', "result": {"foo":'"$cli_params"'}}'
done

We restart the plugin

â—‰ tony@tony:~/lnroom:
$ l1-cli plugin stop $(pwd)/myplugin.bash
...
â—‰ tony@tony:~/lnroom:
$ l1-cli plugin start $(pwd)/myplugin.bash
...

and we call mymethod with no argument, with key/value arguments and with positional arguments which gives us the following:

â—‰ tony@tony:~/lnroom:
$ l1-cli mymethod
{
   "foo": []
}
â—‰ tony@tony:~/lnroom:
$ l1-cli -k mymethod foo1=bar1 foo2=bar2
{
   "foo": {
      "foo1": "bar1",
      "foo2": "bar2"
   }
}
â—‰ tony@tony:~/lnroom:
$ l1-cli mymethod bar1 bar2
{
   "foo": [
      "bar1",
      "bar2"
   ]
}

node id and JSON-RPC over Unix socket

Now let's retrieve the node id of the node running the plugin.

In our terminal using lightning-cli client, we can get the node by calling getinfo command like this:

â—‰ tony@tony:~/lnroom:
$ l1-cli getinfo | jq .id -r
024da50dc4c50f06c2c00c5a2363fc0ccdd28b0d30323d290de43415d63445a830

We could also use nc utility to open a Unix socket connection with the node l1 like this

â—‰ tony@tony:~/lnroom:
$ nc -U /tmp/l1-regtest/regtest/lightning-rpc

then inserting the a getinfo request in the terminal

â—‰ tony@tony:~/lnroom:
$ nc -U /tmp/l1-regtest/regtest/lightning-rpc
{"jsonrpc": "2.0", "id": "1", "method": "getinfo", "params": [], "filter": { "id": true }}

and by pressing return we would receive getinfo response from l1 node like this

â—‰ tony@tony:~/lnroom:
$ nc -U /tmp/l1-regtest/regtest/lightning-rpc
{"jsonrpc": "2.0", "id": "1", "method": "getinfo", "params": [], "filter": { "id": true }}
{"jsonrpc":"2.0","id":"1","result":{"id":"024da50dc4c50f06c2c00c5a2363fc0ccdd28b0d30323d290de43415d63445a830"}}

^C

This is something that we can do in our Bash script.

So, in myplugin.bash we use nc with the variable socket_path instead of the file /tmp/l1-regtest/regtest/lightning-rpc, we do a process substition (<(...)) that we redirect (<) to the read command in order to store the getinfo response in the variable JSON_1:

...
# i/o loop

while read -r JSON; do
    read -r _
    # echo "$JSON" >> /tmp/myplugin
    id=$(echo "$JSON" | jq .id)
    cli_params=$(echo "$JSON" | jq .params)
    read -r JSON_1 < <(echo '{"jsonrpc": "2.0", "id": "1", "method": "getinfo", "params": [], "filter": { "id": true }}' | nc -U $socket_path)
    node_id=$(echo "$JSON_1" | jq .result.id)
    echo '{"jsonrpc": "2.0", "id": '"$id"', "result": {"foo":'"$JSON_1"'}}'
done

We restart the plugin

â—‰ tony@tony:~/lnroom:
$ l1-cli plugin stop $(pwd)/myplugin.bash
...
â—‰ tony@tony:~/lnroom:
$ l1-cli plugin start $(pwd)/myplugin.bash
...

and we call mymethod to check that this returns the getinfo request filtered in the field foo:

â—‰ tony@tony:~/lnroom:
$ l1-cli mymethod
{
   "foo": {
      "jsonrpc": "2.0",
      "id": "1",
      "result": {
         "id": "024da50dc4c50f06c2c00c5a2363fc0ccdd28b0d30323d290de43415d63445a830"
      }
   }
}

Now we store the node id in the variable node_id and return it the payload of the response:

...
# i/o loop

while read -r JSON; do
    read -r _
    # echo "$JSON" >> /tmp/myplugin
    id=$(echo "$JSON" | jq .id)
    cli_params=$(echo "$JSON" | jq .params)
    read -r JSON_1 < <(echo '{"jsonrpc": "2.0", "id": "1", "method": "getinfo", "params": [], "filter": { "id": true }}' | nc -U $socket_path)
    node_id=$(echo "$JSON_1" | jq .result.id)
    echo '{"jsonrpc": "2.0", "id": '"$id"', "result": {"foo":'"$node_id"'}}'
done

We restart the plugin and call mymethod to check that everything is working as expected:

â—‰ tony@tony:~/lnroom:
$ l1-cli plugin stop $(pwd)/myplugin.bash
...
â—‰ tony@tony:~/lnroom:
$ l1-cli plugin start $(pwd)/myplugin.bash
...
â—‰ tony@tony:~/lnroom:
$ l1-cli mymethod
{
   "foo": "024da50dc4c50f06c2c00c5a2363fc0ccdd28b0d30323d290de43415d63445a830"
}

Finally, we modify myplugin.bash to returns the node id, the startup options and the cli parameters in the payload of the repsonses to mymethod requests:

...
# i/o loop

while read -r JSON; do
    read -r _
    # echo "$JSON" >> /tmp/myplugin
    id=$(echo "$JSON" | jq .id)
    cli_params=$(echo "$JSON" | jq .params)
    read -r JSON_1 < <(echo '{"jsonrpc": "2.0", "id": "1", "method": "getinfo", "params": [], "filter": { "id": true }}' | nc -U $socket_path)
    node_id=$(echo "$JSON_1" | jq .result.id)
    echo '{"jsonrpc": "2.0", "id": '"$id"', "result": {"id": '"$node_id"', "options": {"foo_opt": '"$foo_opt"'}, "cli_params": '"$cli_params"'}}'
done

We check that this works by restarting the plugin and calling mymethod method with no arguments and key/value arguments:

â—‰ tony@tony:~/lnroom:
$ l1-cli plugin stop $(pwd)/myplugin.bash
...
â—‰ tony@tony:~/lnroom:
$ l1-cli plugin start $(pwd)/myplugin.bash
...
â—‰ tony@tony:~/lnroom:
$ l1-cli mymethod
{
   "id": "024da50dc4c50f06c2c00c5a2363fc0ccdd28b0d30323d290de43415d63445a830",
   "options": {
      "foo_opt": "bar"
   },
   "cli_params": []
}
â—‰ tony@tony:~/lnroom:
$ l1-cli -k mymethod foo1=bar1 foo2=bar2
{
   "id": "024da50dc4c50f06c2c00c5a2363fc0ccdd28b0d30323d290de43415d63445a830",
   "options": {
      "foo_opt": "bar"
   },
   "cli_params": {
      "foo1": "bar1",
      "foo2": "bar2"
   }
}

Last thing! Does foo_opt startup option get set correctly? Let's find out by restarting the plugin with foo_opt set to BAR

â—‰ tony@tony:~/lnroom:
$ l1-cli plugin stop $(pwd)/myplugin.bash
...
â—‰ tony@tony:~/lnroom:
$ l1-cli -k plugin subcommand=start plugin=$(pwd)/myplugin.bash foo_opt=BAR
...

and calling mymethod method:

â—‰ tony@tony:~/lnroom:
$ l1-cli -k mymethod foo1=bar1 foo2=bar2
{
   "id": "024da50dc4c50f06c2c00c5a2363fc0ccdd28b0d30323d290de43415d63445a830",
   "options": {
      "foo_opt": "BAR"
   },
   "cli_params": {
      "foo1": "bar1",
      "foo2": "bar2"
   }
}

Everything is OK.

We are done!

Terminal session

We ran the following commands in this order:

$ ./setup.sh
$ source lightning/contrib/startup_regtest.sh
$ start_ln
$ alias l1-cli
$ l1-cli plugin start $(pwd)/myplugin.bash
$ l1-cli plugin start $(pwd)/myplugin.bash
$ l1-cli plugin start $(pwd)/myplugin.bash
$ l1-cli plugin start $(pwd)/myplugin.bash
$ l1-cli plugin start $(pwd)/myplugin.bash
$ l1-cli plugin start $(pwd)/myplugin.bash
$ l1-cli plugin start $(pwd)/myplugin.bash
$ l1-cli plugin start $(pwd)/myplugin.bash
$ l1-cli plugin start $(pwd)/myplugin.bash
$ l1-cli plugin start $(pwd)/myplugin.bash
$ ps -ax | rg myplugin
$ l1-cli plugin start $(pwd)/myplugin.bash
$ ps -ax | rg myplugin
$ l1-cli mymethod
$ l1-cli -k mymethod foo1=bar1 foo2=bar2
$ l1-cli mymethod bar1 bar2
$ l1-cli plugin stop $(pwd)/myplugin.bash
$ l1-cli plugin start $(pwd)/myplugin.bash
$ l1-cli mymethod
$ l1-cli plugin stop $(pwd)/myplugin.bash
$ l1-cli plugin start $(pwd)/myplugin.bash
$ l1-cli mymethod
$ l1-cli -k mymethod foo1=bar1 foo2=bar2
$ l1-cli mymethod bar1 bar2
$ l1-cli getinfo | jq .id -r
$ nc -U /tmp/l1-regtest/regtest/lightning-rpc
$ l1-cli plugin stop $(pwd)/myplugin.bash
$ l1-cli plugin start $(pwd)/myplugin.bash
$ l1-cli mymethod
$ l1-cli plugin stop $(pwd)/myplugin.bash
$ l1-cli plugin start $(pwd)/myplugin.bash
$ l1-cli mymethod
$ l1-cli plugin stop $(pwd)/myplugin.bash
$ l1-cli plugin start $(pwd)/myplugin.bash
$ l1-cli mymethod
$ l1-cli -k mymethod foo1=bar1 foo2=bar2
$ l1-cli plugin stop $(pwd)/myplugin.bash
$ l1-cli -k plugin subcommand=start plugin=$(pwd)/myplugin.bash foo_opt=BAR
$ l1-cli -k mymethod foo1=bar1 foo2=bar2

And below you can read the terminal session (command lines and outputs):

â—‰ tony@tony:~/lnroom:
$ ./setup.sh
Ubuntu 22.04.2 LTS
lightningd v23.02.2
GNU bash, version 5.1.16(1)-release (x86_64-pc-linux-gnu)
â—‰ tony@tony:~/lnroom:
$ source lightning/contrib/startup_regtest.sh
lightning-cli is hashed (/usr/local/bin/lightning-cli)
lightningd is /usr/local/bin/lightningd
Useful commands:
  start_ln 3: start three nodes, l1, l2, l3
  connect 1 2: connect l1 and l2
  fund_nodes: connect all nodes with channels, in a row
  stop_ln: shutdown
  destroy_ln: remove ln directories
â—‰ tony@tony:~/lnroom:
$ start_ln
Bitcoin Core starting
awaiting bitcoind...
Making "default" bitcoind wallet.
[1] 1139027
[2] 1139063
WARNING: eatmydata not found: instal it for faster testing
Commands:
        l1-cli, l1-log,
        l2-cli, l2-log,
        bt-cli, stop_ln, fund_nodes
â—‰ tony@tony:~/lnroom:
$ alias l1-cli
alias l1-cli='lightning-cli --lightning-dir=/tmp/l1-regtest'
â—‰ tony@tony:~/lnroom:
$ l1-cli plugin start $(pwd)/myplugin.bash
{
   "code": -3,
   "message": "/home/tony/lnroom/myplugin.bash: exited before replying to getmanifest"
}
â—‰ tony@tony:~/lnroom:
$ l1-cli plugin start $(pwd)/myplugin.bash
{
   "code": -3,
   "message": "/home/tony/lnroom/myplugin.bash: exited before replying to getmanifest"
}
â—‰ tony@tony:~/lnroom:
$ l1-cli plugin start $(pwd)/myplugin.bash
{
   "code": -3,
   "message": "/home/tony/lnroom/myplugin.bash: exited before replying to getmanifest"
}
â—‰ tony@tony:~/lnroom:
$ l1-cli plugin start $(pwd)/myplugin.bash
{
   "code": -3,
   "message": "/home/tony/lnroom/myplugin.bash: exited before replying to getmanifest"
}
â—‰ tony@tony:~/lnroom:
$ l1-cli plugin start $(pwd)/myplugin.bash
{
   "code": -3,
   "message": "/home/tony/lnroom/myplugin.bash: exited before replying to getmanifest"
}
â—‰ tony@tony:~/lnroom:
$ l1-cli plugin start $(pwd)/myplugin.bash
{
   "code": -3,
   "message": "/home/tony/lnroom/myplugin.bash: exited before replying to init"
}
â—‰ tony@tony:~/lnroom:
$ l1-cli plugin start $(pwd)/myplugin.bash
{
   "code": -3,
   "message": "/home/tony/lnroom/myplugin.bash: exited before replying to init"
}
â—‰ tony@tony:~/lnroom:
$ l1-cli plugin start $(pwd)/myplugin.bash
{
   "code": -3,
   "message": "/home/tony/lnroom/myplugin.bash: exited before replying to init"
}
â—‰ tony@tony:~/lnroom:
$ l1-cli plugin start $(pwd)/myplugin.bash
{
   "code": -3,
   "message": "/home/tony/lnroom/myplugin.bash: exited before replying to init"
}
â—‰ tony@tony:~/lnroom:
$ l1-cli plugin start $(pwd)/myplugin.bash
{
   "command": "start",
   "plugins": [
      {
         "name": "/usr/local/libexec/c-lightning/plugins/autoclean",
         "active": true,
         "dynamic": false
      },
      {
         "name": "/usr/local/libexec/c-lightning/plugins/chanbackup",
         "active": true,
         "dynamic": false
      },
      {
         "name": "/usr/local/libexec/c-lightning/plugins/bcli",
         "active": true,
         "dynamic": false
      },
      {
         "name": "/usr/local/libexec/c-lightning/plugins/commando",
         "active": true,
         "dynamic": false
      },
      {
         "name": "/usr/local/libexec/c-lightning/plugins/funder",
         "active": true,
         "dynamic": true
      },
      {
         "name": "/usr/local/libexec/c-lightning/plugins/topology",
         "active": true,
         "dynamic": false
      },
      {
         "name": "/usr/local/libexec/c-lightning/plugins/keysend",
         "active": true,
         "dynamic": false
      },
      {
         "name": "/usr/local/libexec/c-lightning/plugins/offers",
         "active": true,
         "dynamic": true
      },
      {
         "name": "/usr/local/libexec/c-lightning/plugins/pay",
         "active": true,
         "dynamic": true
      },
      {
         "name": "/usr/local/libexec/c-lightning/plugins/txprepare",
         "active": true,
         "dynamic": true
      },
      {
         "name": "/usr/local/libexec/c-lightning/plugins/spenderp",
         "active": true,
         "dynamic": false
      },
      {
         "name": "/usr/local/libexec/c-lightning/plugins/sql",
         "active": true,
         "dynamic": true
      },
      {
         "name": "/usr/local/libexec/c-lightning/plugins/bookkeeper",
         "active": true,
         "dynamic": false
      },
      {
         "name": "/home/tony/lnroom/myplugin.bash",
         "active": true,
         "dynamic": true
      }
   ]
}
â—‰ tony@tony:~/lnroom:
$ ps -ax | rg myplugin
1139988 pts/1    S+     0:00 rg myplugin
â—‰ tony@tony:~/lnroom:
$ l1-cli plugin start $(pwd)/myplugin.bash
{
   "command": "start",
   "plugins": [...]
}
â—‰ tony@tony:~/lnroom:
$ ps -ax | rg myplugin
1140059 pts/1    S      0:00 /usr/bin/bash /home/tony/lnroom/myplugin.bash
1140078 pts/1    S+     0:00 rg myplugin
â—‰ tony@tony:~/lnroom:
$ l1-cli mymethod
^C
â—‰ tony@tony:~/lnroom:
$ l1-cli -k mymethod foo1=bar1 foo2=bar2
^C
â—‰ tony@tony:~/lnroom:
$ l1-cli mymethod bar1 bar2
^C
â—‰ tony@tony:~/lnroom:
$ l1-cli plugin stop $(pwd)/myplugin.bash
{
   "command": "stop",
   "result": "Successfully stopped myplugin.bash."
}
â—‰ tony@tony:~/lnroom:
$ l1-cli plugin start $(pwd)/myplugin.bash
{
   "command": "start",
   "plugins": [...]
}
â—‰ tony@tony:~/lnroom:
$ l1-cli mymethod
{
   "foo": "bar"
}
â—‰ tony@tony:~/lnroom:
$ l1-cli plugin stop $(pwd)/myplugin.bash
{
   "command": "stop",
   "result": "Successfully stopped myplugin.bash."
}
â—‰ tony@tony:~/lnroom:
$ l1-cli plugin start $(pwd)/myplugin.bash
{
   "command": "start",
   "plugins": [...]
}
â—‰ tony@tony:~/lnroom:
$ l1-cli mymethod
{
   "foo": []
}
â—‰ tony@tony:~/lnroom:
$ l1-cli -k mymethod foo1=bar1 foo2=bar2
{
   "foo": {
      "foo1": "bar1",
      "foo2": "bar2"
   }
}
â—‰ tony@tony:~/lnroom:
$ l1-cli mymethod bar1 bar2
{
   "foo": [
      "bar1",
      "bar2"
   ]
}
â—‰ tony@tony:~/lnroom:
$ l1-cli getinfo | jq .id -r
024da50dc4c50f06c2c00c5a2363fc0ccdd28b0d30323d290de43415d63445a830
â—‰ tony@tony:~/lnroom:
$ nc -U /tmp/l1-regtest/regtest/lightning-rpc
{"jsonrpc": "2.0", "id": "1", "method": "getinfo", "params": [], "filter": { "id": true }}
{"jsonrpc":"2.0","id":"1","result":{"id":"024da50dc4c50f06c2c00c5a2363fc0ccdd28b0d30323d290de43415d63445a830"}}

^C
â—‰ tony@tony:~/lnroom:
$ l1-cli plugin stop $(pwd)/myplugin.bash
{
   "command": "stop",
   "result": "Successfully stopped myplugin.bash."
}
â—‰ tony@tony:~/lnroom:
$ l1-cli plugin start $(pwd)/myplugin.bash
{
   "command": "start",
   "plugins": [...]
}
â—‰ tony@tony:~/lnroom:
$ l1-cli mymethod
{
   "foo": {
      "jsonrpc": "2.0",
      "id": "1",
      "result": {
         "id": "024da50dc4c50f06c2c00c5a2363fc0ccdd28b0d30323d290de43415d63445a830"
      }
   }
}
â—‰ tony@tony:~/lnroom:
$ l1-cli plugin stop $(pwd)/myplugin.bash
{
   "command": "stop",
   "result": "Successfully stopped myplugin.bash."
}
â—‰ tony@tony:~/lnroom:
$ l1-cli plugin start $(pwd)/myplugin.bash
{
   "command": "start",
   "plugins": [...]
}
â—‰ tony@tony:~/lnroom:
$ l1-cli mymethod
{
   "foo": "024da50dc4c50f06c2c00c5a2363fc0ccdd28b0d30323d290de43415d63445a830"
}
â—‰ tony@tony:~/lnroom:
$ l1-cli plugin stop $(pwd)/myplugin.bash
{
   "command": "stop",
   "result": "Successfully stopped myplugin.bash."
}
â—‰ tony@tony:~/lnroom:
$ l1-cli plugin start $(pwd)/myplugin.bash
{
   "command": "start",
   "plugins": [...]
}
â—‰ tony@tony:~/lnroom:
$ l1-cli mymethod
{
   "id": "024da50dc4c50f06c2c00c5a2363fc0ccdd28b0d30323d290de43415d63445a830",
   "options": {
      "foo_opt": "bar"
   },
   "cli_params": []
}
â—‰ tony@tony:~/lnroom:
$ l1-cli -k mymethod foo1=bar1 foo2=bar2
{
   "id": "024da50dc4c50f06c2c00c5a2363fc0ccdd28b0d30323d290de43415d63445a830",
   "options": {
      "foo_opt": "bar"
   },
   "cli_params": {
      "foo1": "bar1",
      "foo2": "bar2"
   }
}
â—‰ tony@tony:~/lnroom:
$ l1-cli plugin stop $(pwd)/myplugin.bash
{
   "command": "stop",
   "result": "Successfully stopped myplugin.bash."
}
â—‰ tony@tony:~/lnroom:
$ l1-cli -k plugin subcommand=start plugin=$(pwd)/myplugin.bash foo_opt=BAR
{
   "command": "start",
   "plugins": [...]
}
â—‰ tony@tony:~/lnroom:
$ l1-cli -k mymethod foo1=bar1 foo2=bar2
{
   "id": "024da50dc4c50f06c2c00c5a2363fc0ccdd28b0d30323d290de43415d63445a830",
   "options": {
      "foo_opt": "BAR"
   },
   "cli_params": {
      "foo1": "bar1",
      "foo2": "bar2"
   }
}

Source code

myplugin.bash

#!/usr/bin/bash

set -e

[[ -e /tmp/myplugin ]] && rm /tmp/myplugin

# getmanifest

read -r JSON
read -r _
# echo "$JSON" >> /tmp/myplugin
id=$(echo "$JSON" | jq .id)
echo '{"jsonrpc": "2.0", "id": '"$id"', "result": {"dynamic": true, "options": [{"name": "foo_opt", "type": "string", "default": "bar", "description": "description"}], "rpcmethods": [{"name": "mymethod", "usage": "", "description": "description"}]}}'

# init

read -r JSON
read -r _
# echo "$JSON" >> /tmp/myplugin
id=$(echo "$JSON" | jq .id)
foo_opt=$(echo "$JSON" | jq .params.options.foo_opt)
socket_path=$(echo "$JSON" | jq '.params.configuration."lightning-dir" + "/" + .params.configuration."rpc-file"' -r)

echo '{"jsonrpc": "2.0", "id": '"$id"', "result": {}}'

# i/o loop

while read -r JSON; do
    read -r _
    # echo "$JSON" >> /tmp/myplugin
    id=$(echo "$JSON" | jq .id)
    cli_params=$(echo "$JSON" | jq .params)
    read -r JSON_1 < <(echo '{"jsonrpc": "2.0", "id": "1", "method": "getinfo", "params": [], "filter": { "id": true }}' | nc -U $socket_path)
    node_id=$(echo "$JSON_1" | jq .result.id)
    echo '{"jsonrpc": "2.0", "id": '"$id"', "result": {"id": '"$node_id"', "options": {"foo_opt": '"$foo_opt"'}, "cli_params": '"$cli_params"'}}'
done

setup.sh

#!/usr/bin/env bash

ubuntu=$(lsb_release -ds)
lightningd=$(lightningd --version | xargs printf "lightningd %s\n")
bash_version=$(bash --version | head -n1)

printf "%s\n%s\n%s\n%s" "$ubuntu" "$lightningd" "$bash_version"

Resources