Skip to content

Jobs in Gen3ΒΆ

A job is a finite unit of work that runs to completion (usually within a containerized environment). In Gen3, jobs are generally tasks that load, gather, or dump data. These might be background maintenance cronjobs set to run on a schedule (in our production data commons, an example is the usersync cronjob in Fence that runs to update policies based on the user.yaml), or jobs that are triggered as needed, perhaps manually by users (e.g., PFB creation) or in response to some event in the Gen3 instance (e.g., the ssjdispatcher indexing job when data files are uploaded).

Here is a list of existing Kubernetes jobs in Gen3; note that a few are designed for specific data commons and require a specialized Gen3 setup to work as expected. You can read more about what each job does in this README (it may be missing some more recently-created jobs).

We have highlighted some specific jobs you will likely need early in your Gen3 development in this documentation.