malevich.square.jls

malevich.square.jls.processor(id: str | None = None, finish_msg: str | None = None, drop_internal: bool = True, get_scale_part_all: bool = False)

Denotes a processor of the app.

Processors are core logical units of the app. To understand processors more, see What is Processor?

Parameters:

• id – The id of the processor. If not provided, the name of the function will be used.

• finish_msg – The message to be sent to e-mail (if configured) when the processor finishes.

• drop_internal – Whether to drop the internal collections after the processor finishes.

• get_scale_part_all – Whether to get the scale part of the input collection.

Returns:

The decorated function.

malevich.square.jls.scheme()

Denotes a class as a scheme that might be used in DF/DFS/M/Sink annotations.

The class automatically derives from pydantic.BaseModel.

Example:

@scheme()
class Users:
    name: str
    age: int

@processor()
def process_users(users: DF[Users]):

malevich.square.jls.init(id: str | None = None, enable: bool = True, tl: int | None = None, prepare: bool = False)

Denotes an initialization function.

Initialization functions are called before the app starts. Depending on the configuration, they might be called on task preparation stage or before each task run.

Parameters:

• id – The id of the initialization function. If not provided, the name of the function will be used.

• enable – Whether to enable the initialization function. Defaults to True.

• tl – The time limit of the initialization function (in seconds).

• prepare – Whether to call the initialization function on task preparation stage.