Skip to content

LIVE #5 - How Core Lightning plugins can communicate with each other?

In this live we implement a plugin that emits custom notifications foo to lightningd and another plugin which subscribes to those custom notifications foo. We do it with Python only and also with pyln-client package.

Transcript with corrections and improvements

CLN plugins can talk with each other using CLN's push-based notification mechanism and specifically custom notifications.

In this live, which is divided in two parts of 20 minutes of coding followed by 10 minutes of chat, we'll implement a plugin that emits custom notifications foo to lightningd and another plugin which subscribes to those custom notifications foo.

In the first part we'll write the plugin using Python only. The benefits of doing it without pyln-client first is that it allows us to understand how the system works and that learning can be then applied to other languages (as CLN plugins can be written in any languages).

In the second part we'll write (almost) the same plugins in Python, but this time using pyln-client package.

Custom notifications and subscriptions

Before implementing anything, let's describe the system we'll build today.

We'll write two plugins:

  • foo-emit.py plugin which:
    1. announces the foo custom notification to lightningd and
    2. registers the JSON-RPC command foo-emit (which emits foo custom notifications) to lightningd.
  • foo-subscribe.py plugin:
    1. subscribes to the foo custom notification

Once both plugins are started on a lightning node, each time we call the command foo-emit, foo-emit.py plugin sends a custom notification foo to lightningd, then lightningd forwards that custom notification foo to foo-subscribe.py plugin and finally foo-subscribe.py does something with that custom notification foo:

                  emits `foo`                      forwards `foo`
┌───────────┐ custom notification  ┌──────────┐ custom notification  ┌────────────────┐
│foo-emit.py│--------------------->│lightningd│--------------------->│foo-subscribe.py│
└───────────┘                      └──────────┘                      └────────────────┘

CLN plugin mechanism stages

  1. getmanifest request
  2. init request
  3. io loop

foo-emit.py's reponse to getmanifest request

When the foo-emit.py plugin is started, it receives a getmanifest request from lightningd like this one

{
  "jsonrpc": "2.0",
  "id": 182,
  "method": "getmanifest",
  "params": {
    "allow-deprecated-apis": false
  }
}

and since it wants to register the JSON-RPC foo-emit and to declare the custom notification foo, it just have to relpy to that request with a response that

  1. sets the rpcmethods field of the result member with the foo-emit method and
  2. sets the notifications field of the result member to the array [{"method": "foo"}]

like this:

{
  "jsonrpc": "2.0",
  "id": 182,
  "result": {
    "dynamic": True,
    "options": [],
    "rpcmethods": [{
        "name": "foo-emit",
        "usage": "usage",
        "description": "description"
      }],
    "notifications": [{"method": "foo"}]
  }
}

foo-subscribe.py's reponse to getmanifest request

When the foo-subscribe.py plugin is started, it receives a getmanifest request from lightningd like this one

{
  "jsonrpc": "2.0",
  "id": 196,
  "method": "getmanifest",
  "params": {
    "allow-deprecated-apis": false
  }
}

and since it wants to subscribe to the custom notification foo, it just have to reply to that request with a response that sets the subscriptions field of the result member to the array ["foo"] like this:

{
  "jsonrpc": "2.0",
  "id": 196,
  "result": {
    "dynamic": True,
    "options": [],
    "rpcmethods": [],
    "subscriptions": ["foo"]
  }
}

foo notification as received by foo-subscribe.py

When lightningd forwards the custom notification foo, it wraps the payload of the notification in an object that contains metadata about the notification.

Specifically, when foo-emit.py plugin emits the following custom notification foo to lightningd

{
  "jsonrpc": "2.0",
  "method": "foo",
  "params": {
    "foo": {
      "bar": "baz"
    }
  }
}

foo-subscribe.py plugin receives the following notification forwaded by lightningd with the sender plugin (foo-emit.py) sets in the origin field of the params member:

{
  "jsonrpc": "2.0",
  "method": "foo",
  "params": {
    "origin": "foo-emit.py",
    "payload": {
      "foo": {
        "bar": "baz"
      }
    }
  }
}

Implementation in Python

Start 2 Lightning nodes running on regtest

Let's start two Lightning nodes running on the Bitcoin regtest chain by sourcing the script lightning/contrib/startup_regtest.sh provided in CLN repository and by running the command start_ln:

◉ tony@tony:~/clnlive:
$ source lightning/contrib/startup_regtest.sh
lightning-cli is /usr/local/bin/lightning-cli
lightningd is /usr/local/bin/lightningd
Useful commands:
  start_ln 3: start three nodes, l1, l2, l3
  connect 1 2: connect l1 and l2
  fund_nodes: connect all nodes with channels, in a row
  stop_ln: shutdown
  destroy_ln: remove ln directories
◉ tony@tony:~/clnlive:
$ start_ln
Bitcoin Core starting
awaiting bitcoind...
Making "default" bitcoind wallet.
[1] 324321
[2] 324355
WARNING: eatmydata not found: instal it for faster testing
Commands:
        l1-cli, l1-log,
        l2-cli, l2-log,
        bt-cli, stop_ln, fund_nodes

We can check that l1-cli is just an alias for lightning-cli with the base directory being /tmp/l1-regtest:

◉ tony@tony:~/clnlive:
$ alias l1-cli
alias l1-cli='lightning-cli --lightning-dir=/tmp/l1-regtest'

foo-subscribe.py

  • foo-subscribe.py skeleton

    Instead of writing the plugin from scratch we use parts of the code we wrote during the first live which was about registering JSON-RPC methods to lightningd and understanding CLN plugin system.

    This way we can focus on how to subscribe to notification topics and how to handle notifications.

    So we start with the file myplugin.py containing the following:

    #!/usr/bin/env python

import sys
import json

# getmanifest
request = sys.stdin.readline()
sys.stdin.readline() # "\n"
req_id = json.loads(request)["id"]

manifest = {
    "jsonrpc": "2.0",
    "id": req_id,
    "result": {
        "dynamic": True,
        "options": [],
        "rpcmethods": []
    }
}

sys.stdout.write(json.dumps(manifest))
sys.stdout.flush()

# init
request = sys.stdin.readline()
sys.stdin.readline() # "\n"
req_id = json.loads(request)["id"]

init = {
    "jsonrpc": "2.0",
    "id": req_id,
    "result": {}
}

sys.stdout.write(json.dumps(init))
sys.stdout.flush()

# io loop

for request in sys.stdin:
    sys.stdin.readline() # "\n"

    In that script, we first receive the getmanifest request from lightningd in our stdin stream, we extract its id and we construct the getmanifest response (the plugin is dynamic, with no startup options and register no JSON-RPC methods) that we send back to lightningd by writing it to our stdout stream:

    ...
# getmanifest
request = sys.stdin.readline()
sys.stdin.readline() # "\n"
req_id = json.loads(request)["id"]

manifest = {
    "jsonrpc": "2.0",
    "id": req_id,
    "result": {
        "dynamic": True,
        "options": [],
        "rpcmethods": []
    }
}

sys.stdout.write(json.dumps(manifest))
sys.stdout.flush()
...

    Then we handle the init request sent by lightningd in our stdin stream:

    ...
# init
request = sys.stdin.readline()
sys.stdin.readline() # "\n"
req_id = json.loads(request)["id"]

init = {
    "jsonrpc": "2.0",
    "id": req_id,
    "result": {}
}

sys.stdout.write(json.dumps(init))
sys.stdout.flush()
...

    And finally we start an I/O loop waiting for incoming request from lightningd:

    ...
# io loop

for request in sys.stdin:
    sys.stdin.readline() # "\n"

  • Subscribe to invoice_creation builtin notification topic

    We want to subscribe to the foo custom notification. But before we do that, let get something similar working that doesn't need the foo custom notifications to "exist" to check that our system is working.

    So, let's subscribe to the builtin notification topic invoice_creation (which is sent each time we create an invoice) like this:

    ...
manifest = {
    "jsonrpc": "2.0",
    "id": req_id,
    "result": {
        "dynamic": True,
        "options": [],
        "rpcmethods": [],
        "subscriptions": ["invoice_creation"]
    }
}
...

    And when we'll receive notifications for that topic, we'll write them into the file /tmp/foo-subscribe like this:

    for request in sys.stdin:
    sys.stdin.readline() # "\n"

    with open("/tmp/foo-subscribe", "a") as f:
        f.write(request)

    Note that we don't need to write any logic because the only notifications we'll ever receive from lightningd are for invoice_creation topic (due to our getmanifest response).

    Note also that since we are handling notifications, we don't send any responses to lightningd unlike what we did before with getmanifest and init requests.

    In our terminal now we can start our plugin, check that we have it running and create an invoice:

    ◉ tony@tony:~/clnlive:
$ l1-cli plugin start $(pwd)/foo-subscribe.py
{
   "command": "start",
   "plugins": [ ...,
      {
         "name": "/home/tony/clnlive/foo-subscribe.py",
         "active": true,
         "dynamic": true
      }
   ]
}
◉ tony@tony:~/clnlive:
$ ps -ax | rg foo
 324613 pts/0    S      0:00 python /home/tony/clnlive/foo-subscribe.py
 324642 pts/0    S+     0:00 rg foo
◉ tony@tony:~/clnlive:
$ l1-cli invoice 0.001btc inv pizza
{
   "payment_hash": "49ac1bd3779ad9d4ad91d258b7e3150e6c682fd319aa23b81aa0d2105d124bd3",
   "expires_at": 1685629291,
   "bolt11": "lnbcrt1m1pjx7mhtsp5k2fd3lf6zwv2zpw3luq3x4t3k5ntg89q89w369enmmkeguxs5yzqpp5fxkph5mhntvaftv36fvt0cc4pekxst7nrx4z8wq65rfpqhgjf0fsdqgwp5h57npxqyjw5qcqp29qyysgqy54hdh9qqaexsh8g2vmpj9c8hzh8edwspkm9vss278dmp22yffpxv3apkfjkq8quru8mp8gtsdqmtf3p8xv8g9v2h8ar8jcvc8mc3mgpj4p5l6",
   "payment_secret": "b292d8fd3a1398a105d1ff01135571b526b41ca0395d1d1733deed9470d0a104",
   "warning_capacity": "Insufficient incoming channel capacity to pay invoice"
}

    As the node l1 is running foo-subscribe.py plugin which subscribes to invoice_creation notifications and write them to the file /tmp/foo-subscribe each time l1 creates an invoice, the file /tmp/foo-subscribe contains the following invoice_creation notification:

    {"jsonrpc":"2.0","method":"invoice_creation","params":{"invoice_creation":{"msat":"100000000msat","preimage":"26d2b1252d066c80560c93fa3d99d35ca4aef03abe4859b9d3fb109d8345f317","label":"inv"}}}

  • Subscribe to foo custom notifications

    Fine, our system is working, now let's replace the subscription to invoice_creation to foo custom notification topic like this

    ...
manifest = {
    "jsonrpc": "2.0",
    "id": req_id,
    "result": {
        "dynamic": True,
        "options": [],
        "rpcmethods": [],
        "subscriptions": ["foo"]
    }
}
...

    and restart foo-subscribe.py plugin:

    ◉ tony@tony:~/clnlive:
$ l1-cli plugin start $(pwd)/foo-subscribe.py
{
   "command": "start",
   "plugins": [...]
}

foo-emit.py

Let's start with the same Python script as before.

To register the JSON-RPC method foo-emit to lightningd we add it in the array rpcmethods of the manifest answer to the getmanifest request like this

...
manifest = {
    "jsonrpc": "2.0",
    "id": req_id,
    "result": {
        "dynamic": True,
        "options": [],
        "rpcmethods": [{
            "name": "foo-emit",
            "usage": "usage",
            "description": "description"
        }]
    }
}
...

and to declare the custom notification foo, we set the notifications field of the result member to the array [{"method": "foo"}] in the manifest answer like this:

manifest = {
    "jsonrpc": "2.0",
    "id": req_id,
    "result": {
        "dynamic": True,
        "options": [],
        "rpcmethods": [{
            "name": "foo-emit",
            "usage": "usage",
            "description": "description"
        }],
        "notifications": [{"method": "foo"}]
    }
}

In our io loop, we are going to receive only foo-emit requests. And each time we receive a foo-emit request we want to send a foo notification to lightningd with its payload being {"foo": {"bar": "baz"}}. To do this we modify foo-emit.py script like this:

# io loop

for request in sys.stdin:
    sys.stdin.readline() # "\n"

    foo_notification = {
        "jsonrpc": "2.0",
        "method": "foo",
        "params": {"foo": {"bar": "baz"}}
    }

    sys.stdout.write(json.dumps(foo_notification))
    sys.stdout.flush()

There something missing in what we wrote, thought it works "almost" correctly. Let's check that script and we'll improve it after.

In our terminal, let's start foo-emit.py plugin and check that we have our two plugins running:

◉ tony@tony:~/clnlive:
$ l1-cli plugin start $(pwd)/foo-emit.py
{
   "command": "start",
   "plugins": [...,
      {
         "name": "/home/tony/clnlive/foo-subscribe.py",
         "active": true,
         "dynamic": true
      },
      {
         "name": "/home/tony/clnlive/foo-emit.py",
         "active": true,
         "dynamic": true
      }
   ]
}
◉ tony@tony:~/clnlive:
$ ps -ax | rg foo
 324716 pts/0    S      0:00 python /home/tony/clnlive/foo-subscribe.py
 324968 pts/0    S      0:00 python /home/tony/clnlive/foo-emit.py
 324991 pts/0    S+     0:00 rg foo

We can now call foo-emit command (which hangs):

◉ tony@tony:~/clnlive:
$ l1-cli foo-emit

This has emitted a foo notification which has been forwarded to foo-subscribe.py plugin which consequently wrote the notification in the file /tmp/foo-subscribe that now contains the following

{"jsonrpc":"2.0","method":"invoice_creation","params":{"invoice_creation":{"msat":"100000000msat","preimage":"26d2b1252d066c80560c93fa3d99d35ca4aef03abe4859b9d3fb109d8345f317","label":"inv"}}}
{"jsonrpc":"2.0","method":"foo","params":{"origin":"foo-emit.py","payload":{"foo": {"bar": "baz"}}}}

and we can prettify the forwarded foo notification like this:

{
  "jsonrpc": "2.0",
  "method": "foo",
  "params": {
    "origin": "foo-emit.py",
    "payload": {
      "foo": {
        "bar": "baz"
      }
    }
  }
}

Why foo-emit command hangs?

This is because when we receive the foo-emit request in the io loop, we notify lightningd with a foo custom notification but we "forget" to reply to lightningd to the foo-emit request. So lightningd waits for a response and the client hangs.

Let's fix that with a meaningful answer like 'foo' notification emited (I should have wrote emitted! anyway) that we send back to lightningd:

# io loop

for request in sys.stdin:
    sys.stdin.readline() # "\n"

    foo_notification = {
        "jsonrpc": "2.0",
        "method": "foo",
        "params": {"foo": {"bar": "baz"}}
    }

    sys.stdout.write(json.dumps(foo_notification))
    sys.stdout.flush()

    req_id = json.loads(request)["id"]

    foo_emit_response = {
        "jsonrpc": "2.0",
        "id": req_id,
        "result": {"notification": "'foo' notification emited"}
    }

    sys.stdout.write(json.dumps(foo_emit_response))
    sys.stdout.flush()

Back to our terminal we can check that the command no longer hangs

◉ tony@tony:~/clnlive:
$ l1-cli plugin start $(pwd)/foo-emit.py
{
   "command": "start",
   "plugins": [...]
}
◉ tony@tony:~/clnlive:
$ l1-cli foo-emit
{
   "notification": "'foo' notification emited"
}

and that the foo notification has been forwaded to foo-subscribe.py plugin which wrote the notification in the /tmp/foo-subscribe again:

{"jsonrpc":"2.0","method":"invoice_creation","params":{"invoice_creation":{"msat":"100000000msat","preimage":"26d2b1252d066c80560c93fa3d99d35ca4aef03abe4859b9d3fb109d8345f317","label":"inv"}}}
{"jsonrpc":"2.0","method":"foo","params":{"origin":"foo-emit.py","payload":{"foo": {"bar": "baz"}}}}
{"jsonrpc":"2.0","method":"foo","params":{"origin":"foo-emit.py","payload":{"foo": {"bar": "baz"}}}}

We are done with that first part.

Chat

  • Can the functionality of one plugin influence the behavior of another plugin?

Implementation in Python with pyln-client

Install pyln-client and restart 2 Lightning nodes running on regtest

Now we are going to write with pyln-client the plugins pyln-emit.py and pyln-subscribe.py which do almost the same thing as we did in the first part.

We start by stopping our nodes and bitcoind using commands provided by the script lightning/contrib/startup_regtest.sh

◉ tony@tony:~/clnlive:
$ stop_ln
Lost connection to the RPC socket.Terminated
Lost connection to the RPC socket.Lost connection to the RPC socket.Lost connection to the RPC socket.Terminated
[1]-  Exit 143                test -f "/tmp/l$i-$network/lightningd-$network.pid" || $EATMYDATA "$LIGHTNINGD" "--lightning-dir=/tmp/l$i-$network"
[2]+  Exit 143                test -f "/tmp/l$i-$network/lightningd-$network.pid" || $EATMYDATA "$LIGHTNINGD" "--lightning-dir=/tmp/l$i-$network"
◉ tony@tony:~/clnlive:
$ destroy_ln
◉ tony@tony:~/clnlive:
$ rm -r ~/.bitcoin/regtest/

then we install pyln-client in a Python virtual environment

◉ tony@tony:~/clnlive:
$ python -m venv .venv
◉ tony@tony:~/clnlive:
$ source .venv/bin/activate
(.venv) ◉ tony@tony:~/clnlive:
$ pip install pyln-client
...

finally we start two Lightning nodes running on the Bitcoin regtest chain and check the alias of the command l1-cli:

(.venv) ◉ tony@tony:~/clnlive:
$ start_ln
Bitcoin Core starting
awaiting bitcoind...
Making "default" bitcoind wallet.
[1] 325771
[2] 325813
WARNING: eatmydata not found: instal it for faster testing
Commands:
        l1-cli, l1-log,
        l2-cli, l2-log,
        bt-cli, stop_ln, fund_nodes
(.venv) ◉ tony@tony:~/clnlive:
$ alias l1-cli
alias l1-cli='lightning-cli --lightning-dir=/tmp/l1-regtest'

pyln-subscribe.py

We want to subscribe to the foo custom notification. But before we do that, let get something similar working that doesn't need the foo custom notifications to "exist" to check that our system is working.

So, let's subscribe to the builtin notification topic invoice_creation and each time we receive that notification we write the invoice informations into the file /tmp/pyln-subscribe:

#!/usr/bin/env python

import json
from pyln.client import Plugin

plugin = Plugin()

@plugin.subscribe("invoice_creation")
def invoice_creation_func(plugin,invoice_creation,**kwargs):
    with open("/tmp/pyln-subscribe", "a") as f:
        f.write(json.dumps(invoice_creation))

plugin.run()

In our terminal now we can start our plugin, check that we have it running and create an invoice:

(.venv) ◉ tony@tony:~/clnlive:
$ l1-cli plugin start $(pwd)/pyln-subscribe.py
{
   "command": "start",
   "plugins": [ ...,
      {
         "name": "/home/tony/clnlive/pyln-subscribe.py",
         "active": true,
         "dynamic": true
      }
   ]
}
(.venv) ◉ tony@tony:~/clnlive:
$ ps -ax | rg pyln
 326051 pts/0    S      0:00 python /home/tony/clnlive/pyln-subscribe.py
 326073 pts/0    S+     0:00 rg pyln
(.venv) ◉ tony@tony:~/clnlive:
$ l1-cli invoice 0.001btc inv-1 pizza
{
   "payment_hash": "dc05a3a17aa300bc5e51a8e357bd2fc3aa300a544fabdad8a33c2463d0af0e42",
   "expires_at": 1685631034,
   "bolt11": "lnbcrt1m1pjx7ad6sp5ynt9cmltnled2ttmd2zjl9yagkmgtul2xpv5j95nky85g55k944spp5msz68gt65vqtchj34r3400f0cw4rqzj5f74a4k9r8sjx8590pepqdqgwp5h57npxqyjw5qcqp29qyysgqe9xfh49e85p6rven37rvhh0mhau842cr2qwrxhuy9qhmz24jvy6h5ca8lle9w7mwy93qh2tczhxjahatd52hjk6whgvyh0clfuwe62cqs2lyy3",
   "payment_secret": "24d65c6feb9ff2d52d7b6a852f949d45b685f3ea3059491693b10f4452962d6b",
   "warning_capacity": "Insufficient incoming channel capacity to pay invoice"
}

As the node l1 is running pyln-subscribe.py plugin which subscribes to invoice_creation notifications and write them to the file /tmp/pyln-subscribe each time l1 creates an invoice, the file /tmp/pyln-subscribe contains the following invoice_creation notification:

{"msat": "100000000msat", "preimage": "4d8ce6a06146e0b6d4a5857f77ba312a3adfaebebf8d7068efd98428d6bfdd75", "label": "inv-1"}

Fine, our system is working, now let's replace the subscription to invoice_creation to foo custom notification topic like this

#!/usr/bin/env python

import json
from pyln.client import Plugin

plugin = Plugin()

@plugin.subscribe("foo")
def foo_func(plugin,payload,**kwargs):
    with open("/tmp/pyln-subscribe", "a") as f:
        f.write(json.dumps(payload))

plugin.run()

and restart pyln-subscribe.py plugin and check that it is running:

(.venv) ◉ tony@tony:~/clnlive:
$ l1-cli plugin start $(pwd)/pyln-subscribe.py
{
   "command": "start",
   "plugins": [ ...,
      {
         "name": "/home/tony/clnlive/pyln-subscribe.py",
         "active": true,
         "dynamic": true
      }
   ]
}
(.venv) ◉ tony@tony:~/clnlive:
$ ps -ax | rg pyln
 326233 pts/0    S      0:00 python /home/tony/clnlive/pyln-subscribe.py
 326255 pts/0    S+     0:00 rg pyln

pyln-emit.py

We can use notify method of the class Plugin to send notifications to lightningd. The first argument is the method of the notification (remember that a JSON-RPC notification is a JSON-RPC request without any id member) and the second is the payload (what goes into the params of the request).

With that said we can register foo-emite JSON-RPC method to lightningd that sends foo custom notifications to lightningd with the payload being {"foo": {"bar": "baz"}} like this:

#!/usr/bin/env python

from pyln.client import Plugin

plugin = Plugin()

@plugin.method("foo-emit")
def foo_emit_func(plugin):
    plugin.notify("foo", {"foo": {"bar": "baz"}})

plugin.run()

While foo-emit command is well defined, lightningd won't let us send foo notifications without declaring them with add_notification_topic method of the class Plugin like this:

#!/usr/bin/env python

from pyln.client import Plugin

plugin = Plugin()

@plugin.method("foo-emit")
def foo_emit_func(plugin):
    plugin.notify("foo", {"foo": {"bar": "baz"}})

plugin.add_notification_topic("foo")

plugin.run()

In our terminal, we can now start foo-emit.py plugin and check that we have our two plugins running:

(.venv) ◉ tony@tony:~/clnlive:
$ l1-cli plugin start $(pwd)/pyln-emit.py
{
   "command": "start",
   "plugins": [ ...,
      {
         "name": "/home/tony/clnlive/pyln-subscribe.py",
         "active": true,
         "dynamic": true
      },
      {
         "name": "/home/tony/clnlive/pyln-emit.py",
         "active": true,
         "dynamic": true
      }
   ]
}
(.venv) ◉ tony@tony:~/clnlive:
$ ps -ax | rg pyln
 326233 pts/0    S      0:00 python /home/tony/clnlive/pyln-subscribe.py
 326404 pts/0    S      0:00 python /home/tony/clnlive/pyln-emit.py
 326426 pts/0    R+     0:00 rg pyln

Let's run foo-emit command:

(.venv) ◉ tony@tony:~/clnlive:
$ l1-cli foo-emit
null

This has emitted a foo notification which has been forwarded to pyln-subscribe.py plugin which consequently wrote the payload of the notification in the file /tmp/pyln-subscribe that now contains the following

{"msat": "100000000msat", "preimage": "4d8ce6a06146e0b6d4a5857f77ba312a3adfaebebf8d7068efd98428d6bfdd75", "label": "inv-1"}
{"foo": {"bar": "baz"}}

As we did in the previous example, let's the command foo-emit returns a meaningful information by adding a return statment in the function that notifies lightningd:

#!/usr/bin/env python

from pyln.client import Plugin

plugin = Plugin()

@plugin.method("foo-emit")
def foo_emit_func(plugin):
    plugin.notify("foo", {"foo": {"bar": "baz"}})
    return {"notification": "'foo' notification emited"}

plugin.add_notification_topic("foo")

plugin.run()

Back to our terminal, we restart pyln-emit.py plugin and run the command foo-emit:

(.venv) ◉ tony@tony:~/clnlive:
$ l1-cli plugin start $(pwd)/pyln-emit.py
{
   "command": "start",
   "plugins": [...]
}
(.venv) ◉ tony@tony:~/clnlive:
$ l1-cli foo-emit
{
   "notification": "'foo' notification emited"
}

It worked correctly and the file /tmp/pyln-subscribe is now:

{"msat": "100000000msat", "preimage": "4d8ce6a06146e0b6d4a5857f77ba312a3adfaebebf8d7068efd98428d6bfdd75", "label": "inv-1"}
{"foo": {"bar": "baz"}}
{"foo": {"bar": "baz"}}

Finally, we also write the sender of the notification in the file /tmp/pyln-subscribe by modifying how pyln-subscribe.py handles foo custom notifications:

#!/usr/bin/env python

import json
from pyln.client import Plugin

plugin = Plugin()

@plugin.subscribe("foo")
def foo_func(plugin,origin,payload,**kwargs):
    params = {
        "origin": origin,
        "payload": payload
    }
    with open("/tmp/pyln-subscribe", "a") as f:
        f.write(json.dumps(params))

plugin.run()

Back to our terminal, we restart pyln-emit.py plugin and run the command foo-emit:

(.venv) ◉ tony@tony:~/clnlive:
$ l1-cli plugin start $(pwd)/pyln-subscribe.py
{
   "command": "start",
   "plugins": [...]
}
(.venv) ◉ tony@tony:~/clnlive:
$ l1-cli foo-emit
{
   "notification": "'foo' notification emited"
}

It worked correctly and the file /tmp/pyln-subscribe is now:

{"msat": "100000000msat", "preimage": "4d8ce6a06146e0b6d4a5857f77ba312a3adfaebebf8d7068efd98428d6bfdd75", "label": "inv-1"}
{"foo": {"bar": "baz"}}
{"foo": {"bar": "baz"}}
{"origin": "pyln-emit.py", "payload": {"foo": {"bar": "baz"}}}

We are done with the second part.

Terminal session

We ran the following commands in this order:

$ source lightning/contrib/startup_regtest.sh
$ start_ln
$ alias l1-cli
$ l1-cli plugin start $(pwd)/foo-subscribe.py
$ ps -ax | rg foo
$ l1-cli invoice 0.001btc inv pizza
$ l1-cli plugin start $(pwd)/foo-subscribe.py
$ l1-cli plugin start $(pwd)/foo-emit.py
$ ps -ax | rg foo
$ l1-cli foo-emit
$ l1-cli plugin start $(pwd)/foo-emit.py
$ l1-cli foo-emit
$ stop_ln
$ destroy_ln
$ rm -r ~/.bitcoin/regtest/
$ python -m venv .venv
$ source .venv/bin/activate
$ pip install pyln-client
$ start_ln
$ alias l1-cli
$ l1-cli plugin start $(pwd)/pyln-subscribe.py
$ ps -ax | rg pyln
$ l1-cli invoice 0.001btc inv-1 pizza
$ l1-cli plugin start $(pwd)/pyln-subscribe.py
$ ps -ax | rg pyln
$ l1-cli plugin start $(pwd)/pyln-emit.py
$ ps -ax | rg pyln
$ l1-cli foo-emit
$ l1-cli plugin start $(pwd)/pyln-emit.py
$ l1-cli foo-emit
$ l1-cli plugin start $(pwd)/pyln-subscribe.py
$ l1-cli foo-emit

And below you can read the terminal session (command lines and outputs):

◉ tony@tony:~/clnlive:
$ source lightning/contrib/startup_regtest.sh
lightning-cli is /usr/local/bin/lightning-cli
lightningd is /usr/local/bin/lightningd
Useful commands:
  start_ln 3: start three nodes, l1, l2, l3
  connect 1 2: connect l1 and l2
  fund_nodes: connect all nodes with channels, in a row
  stop_ln: shutdown
  destroy_ln: remove ln directories
◉ tony@tony:~/clnlive:
$ start_ln
Bitcoin Core starting
awaiting bitcoind...
Making "default" bitcoind wallet.
[1] 324321
[2] 324355
WARNING: eatmydata not found: instal it for faster testing
Commands:
        l1-cli, l1-log,
        l2-cli, l2-log,
        bt-cli, stop_ln, fund_nodes
◉ tony@tony:~/clnlive:
$ alias l1-cli
alias l1-cli='lightning-cli --lightning-dir=/tmp/l1-regtest'
◉ tony@tony:~/clnlive:
$ l1-cli plugin start $(pwd)/foo-subscribe.py
{
   "command": "start",
   "plugins": [ ...,
      {
         "name": "/home/tony/clnlive/foo-subscribe.py",
         "active": true,
         "dynamic": true
      }
   ]
}
◉ tony@tony:~/clnlive:
$ ps -ax | rg foo
 324613 pts/0    S      0:00 python /home/tony/clnlive/foo-subscribe.py
 324642 pts/0    S+     0:00 rg foo
◉ tony@tony:~/clnlive:
$ l1-cli invoice 0.001btc inv pizza
{
   "payment_hash": "49ac1bd3779ad9d4ad91d258b7e3150e6c682fd319aa23b81aa0d2105d124bd3",
   "expires_at": 1685629291,
   "bolt11": "lnbcrt1m1pjx7mhtsp5k2fd3lf6zwv2zpw3luq3x4t3k5ntg89q89w369enmmkeguxs5yzqpp5fxkph5mhntvaftv36fvt0cc4pekxst7nrx4z8wq65rfpqhgjf0fsdqgwp5h57npxqyjw5qcqp29qyysgqy54hdh9qqaexsh8g2vmpj9c8hzh8edwspkm9vss278dmp22yffpxv3apkfjkq8quru8mp8gtsdqmtf3p8xv8g9v2h8ar8jcvc8mc3mgpj4p5l6",
   "payment_secret": "b292d8fd3a1398a105d1ff01135571b526b41ca0395d1d1733deed9470d0a104",
   "warning_capacity": "Insufficient incoming channel capacity to pay invoice"
}
◉ tony@tony:~/clnlive:
$ l1-cli plugin start $(pwd)/foo-subscribe.py
{
   "command": "start",
   "plugins": [...]
}
◉ tony@tony:~/clnlive:
$ l1-cli plugin start $(pwd)/foo-emit.py
{
   "command": "start",
   "plugins": [...,
      {
         "name": "/home/tony/clnlive/foo-subscribe.py",
         "active": true,
         "dynamic": true
      },
      {
         "name": "/home/tony/clnlive/foo-emit.py",
         "active": true,
         "dynamic": true
      }
   ]
}
◉ tony@tony:~/clnlive:
$ ps -ax | rg foo
 324716 pts/0    S      0:00 python /home/tony/clnlive/foo-subscribe.py
 324968 pts/0    S      0:00 python /home/tony/clnlive/foo-emit.py
 324991 pts/0    S+     0:00 rg foo
◉ tony@tony:~/clnlive:
$ l1-cli foo-emit
^C
◉ tony@tony:~/clnlive:
$ l1-cli plugin start $(pwd)/foo-emit.py
{
   "command": "start",
   "plugins": [...]
}
◉ tony@tony:~/clnlive:
$ l1-cli foo-emit
{
   "notification": "'foo' notification emited"
}
◉ tony@tony:~/clnlive:
$ stop_ln
Lost connection to the RPC socket.Terminated
Lost connection to the RPC socket.Lost connection to the RPC socket.Lost connection to the RPC socket.Terminated
[1]-  Exit 143                test -f "/tmp/l$i-$network/lightningd-$network.pid" || $EATMYDATA "$LIGHTNINGD" "--lightning-dir=/tmp/l$i-$network"
[2]+  Exit 143                test -f "/tmp/l$i-$network/lightningd-$network.pid" || $EATMYDATA "$LIGHTNINGD" "--lightning-dir=/tmp/l$i-$network"
◉ tony@tony:~/clnlive:
$ destroy_ln
◉ tony@tony:~/clnlive:
$ rm -r ~/.bitcoin/regtest/
◉ tony@tony:~/clnlive:
$ python -m venv .venv
◉ tony@tony:~/clnlive:
$ source .venv/bin/activate
(.venv) ◉ tony@tony:~/clnlive:
$ pip install pyln-client
...
(.venv) ◉ tony@tony:~/clnlive:
$ start_ln
Bitcoin Core starting
awaiting bitcoind...
Making "default" bitcoind wallet.
[1] 325771
[2] 325813
WARNING: eatmydata not found: instal it for faster testing
Commands:
        l1-cli, l1-log,
        l2-cli, l2-log,
        bt-cli, stop_ln, fund_nodes
(.venv) ◉ tony@tony:~/clnlive:
$ alias l1-cli
alias l1-cli='lightning-cli --lightning-dir=/tmp/l1-regtest'
(.venv) ◉ tony@tony:~/clnlive:
$ l1-cli plugin start $(pwd)/pyln-subscribe.py
{
   "command": "start",
   "plugins": [ ...,
      {
         "name": "/home/tony/clnlive/pyln-subscribe.py",
         "active": true,
         "dynamic": true
      }
   ]
}
(.venv) ◉ tony@tony:~/clnlive:
$ ps -ax | rg pyln
 326051 pts/0    S      0:00 python /home/tony/clnlive/pyln-subscribe.py
 326073 pts/0    S+     0:00 rg pyln
(.venv) ◉ tony@tony:~/clnlive:
$ l1-cli invoice 0.001btc inv-1 pizza
{
   "payment_hash": "dc05a3a17aa300bc5e51a8e357bd2fc3aa300a544fabdad8a33c2463d0af0e42",
   "expires_at": 1685631034,
   "bolt11": "lnbcrt1m1pjx7ad6sp5ynt9cmltnled2ttmd2zjl9yagkmgtul2xpv5j95nky85g55k944spp5msz68gt65vqtchj34r3400f0cw4rqzj5f74a4k9r8sjx8590pepqdqgwp5h57npxqyjw5qcqp29qyysgqe9xfh49e85p6rven37rvhh0mhau842cr2qwrxhuy9qhmz24jvy6h5ca8lle9w7mwy93qh2tczhxjahatd52hjk6whgvyh0clfuwe62cqs2lyy3",
   "payment_secret": "24d65c6feb9ff2d52d7b6a852f949d45b685f3ea3059491693b10f4452962d6b",
   "warning_capacity": "Insufficient incoming channel capacity to pay invoice"
}
(.venv) ◉ tony@tony:~/clnlive:
$ l1-cli plugin start $(pwd)/pyln-subscribe.py
{
   "command": "start",
   "plugins": [ ...,
      {
         "name": "/home/tony/clnlive/pyln-subscribe.py",
         "active": true,
         "dynamic": true
      }
   ]
}
(.venv) ◉ tony@tony:~/clnlive:
$ ps -ax | rg pyln
 326233 pts/0    S      0:00 python /home/tony/clnlive/pyln-subscribe.py
 326255 pts/0    S+     0:00 rg pyln
(.venv) ◉ tony@tony:~/clnlive:
$ l1-cli plugin start $(pwd)/pyln-emit.py
{
   "command": "start",
   "plugins": [ ...,
      {
         "name": "/home/tony/clnlive/pyln-subscribe.py",
         "active": true,
         "dynamic": true
      },
      {
         "name": "/home/tony/clnlive/pyln-emit.py",
         "active": true,
         "dynamic": true
      }
   ]
}
(.venv) ◉ tony@tony:~/clnlive:
$ ps -ax | rg pyln
 326233 pts/0    S      0:00 python /home/tony/clnlive/pyln-subscribe.py
 326404 pts/0    S      0:00 python /home/tony/clnlive/pyln-emit.py
 326426 pts/0    R+     0:00 rg pyln
(.venv) ◉ tony@tony:~/clnlive:
$ l1-cli foo-emit
null
(.venv) ◉ tony@tony:~/clnlive:
$ l1-cli plugin start $(pwd)/pyln-emit.py
{
   "command": "start",
   "plugins": [...]
}
(.venv) ◉ tony@tony:~/clnlive:
$ l1-cli foo-emit
{
   "notification": "'foo' notification emited"
}
(.venv) ◉ tony@tony:~/clnlive:
$ l1-cli plugin start $(pwd)/pyln-subscribe.py
{
   "command": "start",
   "plugins": [...]
}
(.venv) ◉ tony@tony:~/clnlive:
$ l1-cli foo-emit
{
   "notification": "'foo' notification emited"
}

Source code

foo-emit.py

#!/usr/bin/env python

import sys
import json

# getmanifest
request = sys.stdin.readline()
sys.stdin.readline() # "\n"
req_id = json.loads(request)["id"]

manifest = {
    "jsonrpc": "2.0",
    "id": req_id,
    "result": {
        "dynamic": True,
        "options": [],
        "rpcmethods": [{
            "name": "foo-emit",
            "usage": "usage",
            "description": "description"
        }],
        "notifications": [{"method": "foo"}]
    }
}

sys.stdout.write(json.dumps(manifest))
sys.stdout.flush()

# init
request = sys.stdin.readline()
sys.stdin.readline() # "\n"
req_id = json.loads(request)["id"]

init = {
    "jsonrpc": "2.0",
    "id": req_id,
    "result": {}
}

sys.stdout.write(json.dumps(init))
sys.stdout.flush()

# io loop

for request in sys.stdin:
    sys.stdin.readline() # "\n"

    foo_notification = {
        "jsonrpc": "2.0",
        "method": "foo",
        "params": {"foo": {"bar": "baz"}}
    }

    sys.stdout.write(json.dumps(foo_notification))
    sys.stdout.flush()

    req_id = json.loads(request)["id"]

    foo_emit_response = {
        "jsonrpc": "2.0",
        "id": req_id,
        "result": {"notification": "'foo' notification emited"}
    }

    sys.stdout.write(json.dumps(foo_emit_response))
    sys.stdout.flush()

foo-subscribe.py

#!/usr/bin/env python

import sys
import json

## getmanifest
request = sys.stdin.readline()
sys.stdin.readline() # "\n"
req_id = json.loads(request)["id"]

manifest = {
    "jsonrpc": "2.0",
    "id": req_id,
    "result": {
        "dynamic": True,
        "options": [],
        "rpcmethods": [],
        "subscriptions": ["foo"]
    }
}

sys.stdout.write(json.dumps(manifest))
sys.stdout.flush()

# init
request = sys.stdin.readline()
sys.stdin.readline() # "\n"
req_id = json.loads(request)["id"]

init = {
    "jsonrpc": "2.0",
    "id": req_id,
    "result": {}
}

sys.stdout.write(json.dumps(init))
sys.stdout.flush()

# io loop

for request in sys.stdin:
    sys.stdin.readline() # "\n"

    with open("/tmp/foo-subscribe", "a") as f:
        f.write(request)

pyln-emit.py

#!/usr/bin/env python

from pyln.client import Plugin

plugin = Plugin()

@plugin.method("foo-emit")
def foo_emit_func(plugin):
    plugin.notify("foo", {"foo": {"bar": "baz"}})
    return {"notification": "'foo' notification emited"}

plugin.add_notification_topic("foo")

plugin.run()

pyln-subscribe.py

#!/usr/bin/env python

import json
from pyln.client import Plugin

plugin = Plugin()

@plugin.subscribe("foo")
def foo_func(plugin,origin,payload,**kwargs):
    params = {
        "origin": origin,
        "payload": payload
    }
    with open("/tmp/pyln-subscribe", "a") as f:
        f.write(json.dumps(params))

plugin.run()

foo-notification-forwarded

{
  "jsonrpc": "2.0",
  "method": "foo",
  "params": {
    "origin": "foo-emit.py",
    "payload": {
      "foo": {
        "bar": "baz"
      }
    }
  }
}

foo-subscribe

{"jsonrpc":"2.0","method":"invoice_creation","params":{"invoice_creation":{"msat":"100000000msat","preimage":"26d2b1252d066c80560c93fa3d99d35ca4aef03abe4859b9d3fb109d8345f317","label":"inv"}}}
{"jsonrpc":"2.0","method":"foo","params":{"origin":"foo-emit.py","payload":{"foo": {"bar": "baz"}}}}
{"jsonrpc":"2.0","method":"foo","params":{"origin":"foo-emit.py","payload":{"foo": {"bar": "baz"}}}}

pyln-subscribe

{"msat": "100000000msat", "preimage": "4d8ce6a06146e0b6d4a5857f77ba312a3adfaebebf8d7068efd98428d6bfdd75", "label": "inv-1"}
{"foo": {"bar": "baz"}}
{"foo": {"bar": "baz"}}
{"origin": "pyln-emit.py", "payload": {"foo": {"bar": "baz"}}}

Resources