Developing plugins#

This page explains how to write a plugin, from a single Python file to an installable project, and how to publish it in the DataLab plugin catalog. For a step-by-step walkthrough, see the tutorial From an idea to the plugin catalog.

Your first plugin in five minutes#

The datalab-plugin command, installed with DataLab as a Python package, creates a working plugin in a single file:

  1. Install DataLab in a Python environment, for example with pip install datalab-platform (see Installation). The standalone version of DataLab does not provide the datalab-plugin command.

  2. Create the plugin, giving its name and a one-line description when asked:

    datalab-plugin create
    

    The command writes a datalab_<name>.py file to the user plugin directory.

  3. Start DataLab, or select Plugins > Reload plugins if it is already running. The Plugins menu now holds a submenu named after the plugin, with a sample processing of the selected signals.

  4. Open the file in your editor and change the sample_processing function, which receives the Y values of a signal as a NumPy array and returns new ones. Select Plugins > Reload plugins to try your changes, without restarting DataLab.

Then, you may:

What is a plugin?#

A plugin is Python code that is automatically loaded by DataLab at startup. It can define new features or modify existing ones.

A plugin defines a class that inherits from datalab.plugins.PluginBase, includes a class attribute named PLUGIN_INFO (an instance of datalab.plugins.PluginInfo) and implements the create_actions method. DataLab finds this class in a plugin directory (see Where to put a plugin?), in one of two forms:

  • A single file whose name starts with datalab_ (e.g. datalab_myplugin.py) and which contains the class. This is the simplest form, recommended to get started: datalab-plugin create creates such a file, ready to use (see Creating a plugin with datalab-plugin create).

  • A folder whose name starts with datalab_ and which holds an __init__.py file (a Python package, e.g. datalab_myplugin/). DataLab loads the folder, then its adapters/desktop.py file when it exists: this is where projects created by datalab-plugin create --project define the class (see Creating a layered plugin project). Otherwise, the class is defined in __init__.py.

Nothing needs to be installed: the file or folder is loaded as it is.

Plugins distributed as installed Python packages, such as the wheels of the plugin catalog, declare their PluginBase subclass through the datalab.plugins entry-point group, without relying on the datalab_ prefix. A plugin found both ways, for example a project that is installed and whose folder is also a plugin directory, is loaded only once.

The PLUGIN_INFO object must define a unique, namespaced id that remains stable when the display name or implementation class changes. DataLab uses this ID for registration and persisted enablement settings. Plugins without an ID remain supported through a module-and-class fallback for backward compatibility, but new plugins should not rely on that fallback.

from datalab.plugins import PluginCapability, PluginInfo

PLUGIN_INFO = PluginInfo(
  id="org.example.my-plugin",
  name="My Plugin",
  version="1.0.0",
  capabilities=(
    PluginCapability.APPLICATION,
    PluginCapability.PROCESSING,
  ),
)

capabilities declares how DataLab may present and consume a plugin. Supported values are PROCESSING, IO, VISUALIZATION and APPLICATION. A plugin may combine several values; domain applications typically declare APPLICATION together with PROCESSING. The plugin configuration dialog shows declared capabilities in a stable order. Existing plugins that omit the field remain valid and are shown without capability labels.

Note

Only files and folders whose names start with datalab_ are scanned for plugins.

DataLab supports three categories of plugins, each with its own purpose and registration mechanism:

  • Processing and visualization plugins Add custom actions for signal or image processing. These may include new computation functions, data visualization tools, or interactive dialogs. Integrated into a dedicated submenu of the “Plugins” menu.

  • Input/Output plugins Define new file formats (read and/or write) handled transparently by DataLab’s I/O framework. These plugins extend compatibility with custom or third-party data formats.

  • HDF5 plugins Special plugins that support HDF5 files with domain-specific tree structures. These allow DataLab to interpret signals or images organized in non-standard ways.

Source organization and distribution#

Source organization, distribution format and functional capabilities are independent concepts. A single-file plugin and a plugin organized as a Python package use the same datalab.plugins.PluginBase API; neither form determines whether it provides processing, input/output, visualization or application features.

  • Single-file plugin: one datalab_<name>.py module, easy to edit and share without a packaging project. This historical form is also the default output of datalab-plugin create; it is not deprecated.

  • Package or project: Python modules and, optionally, resources, tests and packaging metadata. datalab-plugin create --project offers one layered organization, not a mandatory architecture for all plugins.

  • Wheel distribution: an installable .whl archive with standardized metadata and plugin entry points. A wheel can contain a single module or a larger package; it is a distribution format, not a functional plugin category.

Both single-file modules and wheels can be installed with Install from file… or distributed through online and local catalogs. The catalogs do not require the layered project generated by --project. Packaging a single module as a wheel remains possible, but is optional for Desktop.

Development forms and distribution#

Development form

Direct source loading

Managed distribution

Single Python file

datalab_<name>.py in a plugin directory

Raw .py file, or an optional wheel

Python package or project

Package folder, including a project’s src layout

Wheel containing the package and its resources

Both managed formats work in local and online Desktop catalogs. The historical single-file form is not a separate functional plugin category, and its source does not need to be changed merely to list it in a catalog. DataLab-Web support is not implied by a Desktop catalog entry.

A directly loaded source file remains in its development directory. A file installed through DataLab is copied to its managed user storage. A plugin is not a macro: a plugin registers persistent application features, whereas a macro is a script run to automate a task. Supporting the historical single-file form does not mean that obsolete APIs from DataLab v0.20 are automatically supported.

Creating a plugin with datalab-plugin create#

The datalab-plugin command is installed with DataLab as a Python package; the standalone version of DataLab does not provide it. Run it without options to create a single-file plugin:

datalab-plugin create

The command only asks for the plugin name and its description, then writes a datalab_<name>.py file to the user plugin directory (see Where to put a plugin?). Select Plugins > Reload plugins in DataLab, or restart DataLab: the plugin adds its submenu to the Plugins menu, with a sample processing of the selected signals. To implement your own processing, edit the sample_processing function of the file, which receives the Y values of a signal as a NumPy array and returns new ones, then reload the plugins.

Options adapt the generated file:

  • --object-kind image: the sample processing applies to images, and its function receives the image data.

  • --capability: without the processing capability, the submenu holds an action showing a message, to replace with your own.

  • A destination folder, given as argument: it must then be added to the plugin directories (see Advanced tab).

  • --package and --plugin-id: the module name and the plugin ID are otherwise derived from the plugin name. The module name must start with datalab_.

An existing file is never overwritten. To share the plugin, send its file: other users install it with Plugins > Configure plugins… > From file (see From file tab).

Both the raw single-file plugin and a wheel can be published in a Desktop catalog (see Single-file modules in a catalog). To develop a larger plugin with packaging and tests already configured, create a layered project with --project instead (see Creating a layered plugin project).

Creating a layered plugin project#

With --project, the installed datalab-plugin command creates a small, installable project that follows the stable plugin and feature ownership contracts. Run it without other options for an interactive setup:

datalab-plugin create --project

The command asks for the display name and the GitHub account (user or organization) that will host the project, and offers defaults for the Python package, reverse-domain plugin ID, description, and destination. The default plugin ID is io.github.<account>.<name>; org.datalab. IDs are reserved for the DataLab-Platform organization, and org.example. is used when no account is given. For scripts or reproducible setup, pass the values explicitly:

datalab-plugin create --project datalab-camera-characterization \
  --name "Camera Characterization" \
  --github-account DataLab-Platform \
  --package datalab_camera_characterization \
  --plugin-id org.datalab.camera-characterization \
  --description "Characterize scientific cameras" \
  --capability application \
  --capability processing \
  --object-kind image

The generated src-layout project applies the architecture validated by the Camera pilot:

package root (host-independent identity)
├── core
├── workflow
└── adapters
    ├── desktop.py
    └── web.py

core holds host-independent domain behavior, workflow may use DataLab’s headless recipe contracts, and adapters owns host integration. The datalab.plugins entry point targets the Desktop adapter; Web support starts explicitly as unsupported (see Plugins and DataLab-Web). A generated AST test prevents core and workflow from importing GUI, browser, or adapter modules.

The project also includes a stable datalab.plugins.PluginInfo, an owned sample processing when the processing capability is selected, Ruff settings, architecture and contribution documentation, a changelog, a README, and a BSD-3-Clause license. The command refuses to overwrite an existing destination.

The template remains deliberately small. Add domain recipes, DataSet parameters, packaged examples, and specialized CI only when the plugin needs them.

To try the plugin, add the generated directory as a plugin directory (see Advanced tab) and reload the plugins: DataLab loads the plugin class from adapters/desktop.py without installing the project. To validate the project, install it with its test dependencies and run its checks from the generated directory:

python -m pip install -e ".[test]"
python -m pytest
python -m ruff check .

The generated pyproject.toml requires Python 3.10 or later and the current DataLab major version (for example datalab-platform >= 1.4, < 2), and declares the project URLs when the GitHub account is known. The generated .github/workflows/release.yml publishes the plugin without any PyPI account: pushing a tag such as v0.1.0 checks that the tag matches the version of pyproject.toml and of the package, runs the tests, builds the wheel, publishes it in a GitHub release with its SHA-256 checksum and its build provenance attestation, and prints the entry to submit to the DataLab plugin catalog, including the SHA-256 digest of the wheel. A tag such as v0.2.0-rc1 (version 0.2.0rc1) publishes a release candidate as a GitHub pre-release, without catalog entry. Users then install the wheel with Plugins > Configure plugins… > From file (see From file tab). Listing the plugin in the catalog is optional; once listed, it can also be installed from the Catalog tab (see Catalog tab).

Hot-reload workflow for plugin development#

The hot-reload feature is designed to accelerate the plugin development cycle. Here is the recommended workflow:

  1. Start DataLab normally.

  2. Create or edit your plugin file (e.g. datalab_myplugin.py) in one of the plugin directories (e.g. ~/.DataLab_v1/plugins).

  3. In DataLab, use Plugins > Reload plugins to pick up your changes instantly.

  4. Test your plugin actions directly in the running application.

  5. Iterate: edit the file, reload, test — without restarting DataLab.

To selectively enable or disable specific plugins during development, use Plugins > Configure plugins… and its Apply and reload plugins button, so that changes take effect without leaving the dialog.

Plugin API helpers#

Plugins inheriting from datalab.plugins.PluginBase have direct access to useful helpers:

  • self.signalpanel and self.imagepanel: access to panel APIs and action handlers,

  • self.proxy: a datalab.control.proxy.LocalProxy instance for object creation and processing,

  • show_warning, show_error, show_info, ask_yesno: convenience dialog methods,

  • edit_new_signal_parameters and edit_new_image_parameters: helpers for object parameter dialogs.

These helpers simplify plugin code and keep it consistent with DataLab behavior.

Processing plugins should override register_computations() and register each feature with a stable feature_id of the form <plugin_id>:<local_id> and owner_plugin_id=self.plugin_id. DataLab calls this hook after the signal and image panels exist. Owned features are removed automatically when the plugin is disabled, reloaded, or uninstalled. create_actions() may then reference the registered feature by its stable ID.

A registered feature runs a Sigima computation function, decorated with sigima.proc.decorator.computation_function(). For a 1-to-1 feature without parameters, the simplest is to wrap a function processing NumPy arrays with sigima.proc.signal.Wrap1to1Func (signal Y values) or sigima.proc.image.Wrap1to1Func (image data), as in the processing plugin example (see Example: processing plugin). DataLab refuses other functions when they are registered.

Live preview compatibility#

Note

Standard parameterized functions registered with register_1_to_1 can participate in live preview without a custom renderer. They must return a SignalObj or ImageObj, be importable in a spawned process, and accept serializable source/parameter copies. Preview uses a private process and does not publish results to the workspace. It is not a sandbox for filesystem, network or other external side effects. Register functions that must not run speculatively with preview_enabled=False. Serialization or computation failures are reported in the preview; the normal OK path remains available. Unknown custom DataSet.edit implementations and alternate guidata backends are not replaced.

The dedicated preview process may be reused between dialogs. Plugin functions must not assume a fresh Python interpreter for each preview: process-local module state may persist until a preview is cancelled while running, plugins are reloaded, or DataLab exits.

How to develop a plugin?#

The recommended approach to developing a plugin is to derive from an existing example and adapt it to your needs. You can explore the source code in the datalab/plugins/builtin folder, the examples below or the plugins of the DataLab plugin catalog (see Plugins of the catalog).

Note

Most of DataLab’s signal and image processing functionalities have been externalized into a dedicated library called Sigima (https://sigima.readthedocs.io/en/latest/). When developing DataLab plugins, you will typically import and use many Sigima functions and features to perform data processing, analysis, and visualization tasks. Sigima provides a comprehensive set of tools for scientific data manipulation that can be leveraged directly in your plugins.

To develop in your usual Python environment (e.g., with an IDE like Spyder), you can:

  1. Install DataLab in your Python environment, using one of the following methods:

  2. Or add the `datalab` package manually to your Python path:

    • Download the source from the PyPI page,

    • Unzip the archive,

    • Add the parent directory containing the datalab package to your PYTHONPATH (e.g., using the PYTHONPATH Manager in Spyder). Install the project’s dependencies separately.

Note

Even if you’ve installed datalab in your environment, you cannot run the full DataLab application directly from an IDE. You must launch DataLab via the command line or using the installer-created shortcut to properly test your plugin.

Plugins of the catalog#

The plugins of the DataLab plugin catalog are open-source and show complete, tested implementations:

Example: processing plugin#

This example registers a processing feature with stable identity and ownership, then creates an action which dispatches it through the processor registry:

# -*- coding: utf-8 -*-

"""
Custom denoising filter plugin
==============================

This is a simple example of a DataLab image processing plugin.

It is part of the DataLab custom function tutorial.

.. note::

    This plugin is not installed by default. To install it, copy this file to
    your DataLab plugins directory (see `DataLab documentation
    <https://datalab-platform.com/en/features/advanced/plugins.html>`_).
"""

import numpy as np
import scipy.ndimage as spi
import sigima.proc.image as sipi

import datalab.plugins


def weighted_average_denoise(data: np.ndarray) -> np.ndarray:
    """Apply a custom denoising filter to an image.

    This filter averages the pixels in a 5x5 neighborhood, but gives less weight
    to pixels that significantly differ from the central pixel.
    """

    def filter_func(values: np.ndarray) -> float:
        """Filter function"""
        central_pixel = values[len(values) // 2]
        differences = np.abs(values - central_pixel)
        mean_difference = np.mean(differences)
        if mean_difference == 0:
            return central_pixel
        weights = np.exp(-differences / mean_difference)
        return np.average(values, weights=weights)

    return spi.generic_filter(data, filter_func, size=5)


class CustomFilters(datalab.plugins.PluginBase):
    """DataLab Custom Filters Plugin"""

    FEATURE_ID = "org.datalab.examples.custom-filters:weighted-average-denoise"
    PLUGIN_INFO = datalab.plugins.PluginInfo(
        id="org.datalab.examples.custom-filters",
        name="My custom filters",
        version="1.0.0",
        description="This is an example plugin",
        capabilities=(datalab.plugins.PluginCapability.PROCESSING,),
    )

    def register_computations(self) -> None:
        """Register computations owned by this plugin."""
        wrapped_func = sipi.Wrap1to1Func(weighted_average_denoise)
        self.imagepanel.processor.register_1_to_1(
            wrapped_func,
            "Weighted average denoise",
            feature_id=self.FEATURE_ID,
            owner_plugin_id=self.plugin_id,
        )

    def create_actions(self) -> None:
        """Create actions"""
        acth = self.imagepanel.acthandler
        with acth.new_menu(self.PLUGIN_INFO.name):
            acth.action_for(self.FEATURE_ID)

Example: input/output plugin#

Here is a simple example of a plugin that adds new file formats to DataLab.

# Copyright (c) DataLab Platform Developers, BSD 3-Clause license, see LICENSE file.

"""
Image file formats Plugin for DataLab
-------------------------------------

This plugin is an example of DataLab plugin.
It provides image file formats from cameras, scanners, and other acquisition devices.
"""

import struct

import numpy as np
from sigima.io.base import FormatInfo
from sigima.io.image.base import SingleImageFormatBase

# ==============================================================================
# Thales Pixium FXD file format
# ==============================================================================


class FXDFile:
    """Class implementing Thales Pixium FXD Image file reading feature

    Args:
        fname (str): path to FXD file
        debug (bool): debug mode
    """

    HEADER = "<llllllffl"

    def __init__(self, fname: str = None, debug: bool = False) -> None:
        self.__debug = debug
        self.file_format = None  # long
        self.nbcols = None  # long
        self.nbrows = None  # long
        self.nbframes = None  # long
        self.pixeltype = None  # long
        self.quantlevels = None  # long
        self.maxlevel = None  # float
        self.minlevel = None  # float
        self.comment_length = None  # long
        self.fname = None
        self.data = None
        if fname is not None:
            self.load(fname)

    def __repr__(self) -> str:
        """Return a string representation of the object"""
        info = (
            ("Image width", f"{self.nbcols:d}"),
            ("Image Height", f"{self.nbrows:d}"),
            ("Frame number", f"{self.nbframes:d}"),
            ("File format", f"{self.file_format:d}"),
            ("Pixel type", f"{self.pixeltype:d}"),
            ("Quantlevels", f"{self.quantlevels:d}"),
            ("Min. level", f"{self.minlevel:f}"),
            ("Max. level", f"{self.maxlevel:f}"),
            ("Comment length", f"{self.comment_length:d}"),
        )
        desc_len = max(len(d) for d in list(zip(*info))[0]) + 3
        res = ""
        for description, value in info:
            res += ("{:" + str(desc_len) + "}{}\n").format(description + ": ", value)

        res = object.__repr__(self) + "\n" + res
        return res

    def load(self, fname: str) -> None:
        """Load header and image pixel data

        Args:
            fname (str): path to FXD file
        """
        with open(fname, "rb") as data_file:
            header_s = struct.Struct(self.HEADER)
            record = data_file.read(9 * 4)
            unpacked_rec = header_s.unpack(record)
            (
                self.file_format,
                self.nbcols,
                self.nbrows,
                self.nbframes,
                self.pixeltype,
                self.quantlevels,
                self.maxlevel,
                self.minlevel,
                self.comment_length,
            ) = unpacked_rec
            if self.__debug:
                print(unpacked_rec)
                print(self)
            data_file.seek(128 + self.comment_length)
            if self.pixeltype == 0:
                size, dtype = 4, np.float32
            elif self.pixeltype == 1:
                size, dtype = 2, np.uint16
            elif self.pixeltype == 2:
                size, dtype = 1, np.uint8
            else:
                raise NotImplementedError(f"Unsupported pixel type: {self.pixeltype}")
            block = data_file.read(self.nbrows * self.nbcols * size)
        data = np.frombuffer(block, dtype=dtype)
        self.data = data.reshape(self.nbrows, self.nbcols)


class FXDImageFormat(SingleImageFormatBase):
    """Object representing Thales Pixium (FXD) image file type"""

    FORMAT_INFO = FormatInfo(
        name="Thales Pixium",
        extensions="*.fxd",
        readable=True,
        writeable=False,
    )

    @staticmethod
    def read_data(filename: str) -> np.ndarray:
        """Read data and return it

        Args:
            filename (str): path to FXD file

        Returns:
            np.ndarray: image data
        """
        fxd_file = FXDFile(filename)
        return fxd_file.data


# ==============================================================================
# Dürr NDT XYZ file format
# ==============================================================================


class XYZImageFormat(SingleImageFormatBase):
    """Object representing Dürr NDT XYZ image file type"""

    FORMAT_INFO = FormatInfo(
        name="Dürr NDT",
        extensions="*.xyz",
        readable=True,
        writeable=True,
    )

    @staticmethod
    def read_data(filename: str) -> np.ndarray:
        """Read data and return it

        Args:
            filename (str): path to XYZ file

        Returns:
            np.ndarray: image data
        """
        with open(filename, "rb") as fdesc:
            cols = int(np.fromfile(fdesc, dtype=np.uint16, count=1)[0])
            rows = int(np.fromfile(fdesc, dtype=np.uint16, count=1)[0])
            arr = np.fromfile(fdesc, dtype=np.uint16, count=cols * rows)
            arr = arr.reshape((rows, cols))
        return np.fliplr(arr)

    @staticmethod
    def write_data(filename: str, data: np.ndarray) -> None:
        """Write data to file

        Args:
            filename: File name
            data: Image array data
        """
        data = np.fliplr(data)
        with open(filename, "wb") as fdesc:
            fdesc.write(np.array(data.shape[1], dtype=np.uint16).tobytes())
            fdesc.write(np.array(data.shape[0], dtype=np.uint16).tobytes())
            fdesc.write(data.tobytes())

Example templates used by the test suite#

DataLab also provides plugin templates used by integration tests in datalab/tests/features/plugins/templates. They are useful as development references for:

  • basic valid plugin structure,

  • processing feature owned by a plugin (register_computations),

  • nested plugin menus,

  • plugins with dialog actions,

  • plugins with many actions,

  • plugins with long descriptions.

The corresponding feature tests are located in datalab/tests/features/plugins/plugins_app_test.py and cover plugin lifecycle, hot-reload behavior, error handling, duplicate names, and configuration filtering.

Other examples#

Other examples of plugins can be found in the plugins/examples directory of the DataLab source code (explore here on GitHub).

Publishing a plugin in the catalog#

Any author may list a plugin in the DataLab plugin catalog, so that DataLab users find it and install it from the Catalog tab (see Catalog tab). No PyPI account is needed:

  1. Host the source code in a public GitHub repository, under an OSI-approved open-source license.

  2. Attach the plugin file to a GitHub release: the unchanged datalab_<name>.py file of a single-file plugin, or the wheel of a plugin project, which the release workflow generated by datalab-plugin create --project builds and publishes.

  3. Open a pull request adding one YAML file, which describes the plugin and its release, to the catalog repository. Its automatic checks download the plugin file, verify its SHA-256 digest and inspect it without running it.

Once the pull request is merged, the plugin appears on the catalog page and in the Catalog tab of DataLab, marked Community: the Official mark is reserved for the plugins of the DataLab team. The tutorial From an idea to the plugin catalog details each step, and the contribution guide gives the complete rules.

Plugins and DataLab-Web#

DataLab-Web, the browser-native edition of the platform, provides its own implementation of the datalab.plugins.PluginBase API, which differs from the one of DataLab Desktop. A simple processing plugin using the shared API may run in both editions; plugins using host-specific features generally define a distinct class for each of them:

  • One entry point per edition. A wheel declares its Desktop class in the datalab.plugins entry-point group and its DataLab-Web class in the datalab.web_plugins group: the browser only imports the classes of the datalab.web_plugins group. Both entry points may reference the same portable class or distinct classes, which may share the host-independent code of the plugin, such as the core and workflow layers of a project created by datalab-plugin create --project, whose adapters/web.py module starts as unsupported, without Web entry point:

    [project.entry-points."datalab.plugins"]
    my_plugin = "my_package.adapters.desktop:MyDesktopPlugin"
    
    [project.entry-points."datalab.web_plugins"]
    my_plugin = "my_package.adapters.web:MyWebPlugin"
    
  • Processing registration and browser limitations. DataLab-Web supports register_computations() and action_for() for simple processing plugins, including those created by the single-file plugin generator. Desktop welcome page tiles (WELCOME_TILES) are ignored with a warning, and recipe launchers (RECIPE_LAUNCHERS) prevent the plugin from loading. Desktop methods for recipes or examples (start_recipe, try_example, open_example, etc.) are not available. A DataLab-Web class may declare headless recipes, examples and tools (RECIPES, EXAMPLES and TOOLS), which DataLab-Web presents and runs itself.

  • Asynchronous dialogs are specific to DataLab-Web. In the browser, a dialog cannot block the event loop: DataLab-Web plugins use async action callbacks, await param.edit_async(self.main) and the *_async helpers of their base class (such as show_info_async or edit_new_signal_parameters_async). DataLab Desktop does not run async action callbacks (the action then does nothing), and its datalab.plugins.PluginBase class has no *_async helpers: Desktop plugins use synchronous callbacks and helpers.

  • No Qt-only graphical interfaces. Plugins that embed custom Qt widgets or rely on PlotPy interactive tools are desktop-only; their graphical parts have no equivalent in the browser.

  • Execution happens inside the browser (WebAssembly), with no native file-system access beyond the in-memory file system.

The DataLab plugin catalog indicates which of its plugins also support DataLab-Web; the Camera & Detector Characterization, Pulse & Transient Characterization and X-ray Radiography & NDT plugins are examples of plugins with a distinct DataLab-Web class. See the DataLab-Web plugin guide for the plugin API of the browser and for loading plugins in DataLab-Web.

Migrating from v0.20 to v1.0#

If you have existing plugins written for DataLab v0.20, please refer to the migration guide for detailed instructions on updating your plugins to work with DataLab v1.0.