Job Scheduler FAQ

Q: Has Job Scheduler been tested for an external PostgreSQL database in AWS?

There should be no difference where the database is running, as long as it is routable/addressable from the TAP platform. In other words, if PostgreSQL is reachable via a network from TAP, it should work.

Q: How does the security mechanism work for the Job Scheduler?

Submitting jobs occurs on behalf of the TAP user and, therefore, identity is passed to whatever action Oozie is performing. This also applies for the permissions needed to run actions. Depending on the particular action, user permissions are verified and, if they are sufficient, then the action is performed. Please note that not all permissions can be verified during the job submission phase; therefore, you may experience an insufficient permissions problem during runtime.

In particular, job configuration files and imported files are kept in an organization directory on HDFS. Therefore, you must have read/write access to the organization to successfully run jobs that touch those files.

Jobs are scheduled in a yarn queue dedicated for a particular organization on behalf of the user that submitted them. The Oozie server has no concept of multitenancy; therefore, TAP maintains a list of jobs submitted for a particular organization to be able to display them in the UI. We will look at a higher level of integration to enable the resource manager to retrieve this information in the future.

Q: Does the Job Scheduler provide an alert mechanism for job incompletions or failures?

Currently there are no alerts. The scheduler retrieves statuses of submitted jobs and displays them in the UI. Adding alerts for error handling may be possible in the future.