Global

Members

globalReadlineInterface

Description:

  • Internal variables

Source:

Internal variables

Type Definitions

AdapterOption

Description:

  • The option object

Source:
Properties:
Name Type Description
adapter string

The adapter name (Default = 'docker')
cpu number | string

The required cpu cores (might have different meanings, e.g. vcpus; Default = 1)
mem string

The memory size (Default = '4g')
nodes number | string

The computing node number
concurrency number

The concurrency number. If your runtime environment does not support parallel computing such as PBS. Setting to a smaller number is preferred (Default = 1).
batch boolean

If set to true, the standard output/error will be hidden in the standard output/error of the process. Instead, a job counter of BatchStatus will be shown.
updateInterval number

The interval in milliseconds to check the job status. Setting an appropriate number to prevent checking jobs too often or too long. The interval counter starts after each checking process. (Default = 30000).
stdoeMode string

Specifies the ways to monitor job status and recording stdout/stderr.
dockerPath path
projectFolder path

This project folder stores the users' data.
packageFolder path

This package folder stores the package files.
taskLogFolder path

This folder stores the task runtime logs.
sharedFolder path
extFolders Array.<path>

Additional folders that may be required to mount to containers.
cwd path

The current working directory for the adapter process.
workdir path

The working directory of the in-container process.
simplifiedTitle boolean

Set to true if you do not want to print the fancy Bio-Data Processor wellcome title.
setUser boolean

Used to specify if the user id and group id, e.g. '--user uid: gid' for docker adapter.
debug boolean

true/false. When debug is set to true, the adapter can display additional messages about running processes.
env Object

Setting environment variables.
stopOnError boolean

Set to true if you want to stop all other jobs when one of the jobs have errors. (Default = false)
delayInterval number

The interval in milliseconds between job submissions. This delay time is set to avoid submitting too many jobs in a short time.
forceRerun boolean

Ignore the job exitCode and re-run jobs anyway.
retryMode 'normal' | 'eager'

In the eager mode, failed task will be directly pushed back to the task queeu for rerun.
... *

Additional options can be provided for different requirements of adapters.

The option object

Type:
  • Object

BatchStatus

Source:
Properties:
Name Type Description
pending number
queued number
running number
finishing number
exit number
done number
total number
others array
Type:
  • Object

FileHandler

Source:
Properties:
Name Type Description
stdoutFS stream
stderrFS stream
stdoeWatcher *

an object returned by chokidar.
readFileSizes object

An object that stores the sizes of stdout/stderr that the stdoeWatcher has been read.

Properties
Name Type Description
stdout number

The read sizes of stdout in bytes
stderr number

The read sizes of in bytes
Type:
  • Object

JobDefinition

Description:

  • This the format that we store for each task item.

Source:
Properties:
Name Type Description
taskName string

The job name in the task definitions
jobId string

The auto-generated unique id which contains the taskName
pid string

The jobId that is generated from the job running environment. (e.g. pid, pbs job ID, ...)
image string

The image name in the task definitions
exec string

The executable/command for the job in a container
args array

The resolved arguments in the task definitions (These arguments were prepared for different run-time environments).
command string

The executed command for display only.
start number

The job start timestamp in milliseconds since midnight 01, Jan, 1970 UTC
end number

The job start timestamp in milliseconds since midnight 01, Jan, 1970 UTC
stdout path

The file path for standard output
stderr path

The file path for standard error
cpus number

Used for different runtime environments; It depends on the runtime environments; Might have different meanings.
mem string

Used for different runtime environements; It depends on the runtime environments; Might have different meanings.
nodes number

Used for different runtime environements; Might not be used if the adapter only send a task to one single computing node;
exitCode number

The exit code after process exit;
taskArgs array

The arguments before transforming into the runtime arguments. Different adapters will form taskArgs into different runtime arguments, e.g. the docker adapter will make the command begins with docker run ..., while the pbs adapter starts with qsub ....
option AdapterOption

Additional options that initiate the adapter.
proxy Proxy

The proxy object (null if the proxy is not enabled)

This the format that we store for each task item.

Type:
  • Object

ProcessExit

Source:
Properties:
Name Type Description
exitCode number

The process exit code
signal string

The SIGNAL to terminate the process; default is null.
Type:
  • Object

Proxy

Description:

  • The proxy information in a JobDefinition

Source:
Properties:
Name Type Description
protocol string

The protocol of the proxy (currently, only the http protocol is supported)
ip string

The ip address of the proxy
port number

The port number of the proxy
pathRewrite boolean
entryPath string
containerPort number

The port number in the container

The proxy information in a JobDefinition

Type:
  • Object

RunningJob

Source:
Properties:
Name Type Description
runningJobId string
stdoutStream stream | number

Only used in the stdoeMode === 'pipe'; if in the stdoeMode of 'watch', it is used as an object storing the stream loading progress in bytes.
stderrStream stream | number

Only used in the stdoeMode === 'pipe'; if in the stdoeMode of 'watch', it is used as an object storing the stream loading progress in bytes.
jobEmitter eventemitter

This event emitter emits 'finish' or 'error' events when the process is finished or has errors, respectively.
isRunning boolean

a flag to note if the job is queued or not running. If false, it may indicate that the job is sent to a remote resources but queued (not yet started). Used for the job counter display.
Type:
  • Object

TaskAdapterInstance

Source:
Properties:
Name Type Description
getOptions function

use this function to get the BdpTaskAdapter.options object.
addTasks function

calls the BdpTaskAdapter._addTasks functions to add tasks.
run function

execute tasks and exit the Adapter process.
stop function

stop the queue and calling beforeExit() function.
execute function

This is the main function to execute tasks in Big Data Processor. The adapter process wil not exit after this function.
Type:
  • Object