Instant Pentaho Data Integration Kitchen: Explore the world of Pentaho Data Integration command-line tools which will help you use the Kitchen

Create a new empty transformation. To do this, you can perform either of the following steps:

Go to the Transformation properties dialog and define a new transformation parameter called p_country. To do this, perform the following steps:

Select the Design tab from the left view. The list of category folders will appear in the Steps panel. Perform the following steps on them:

Connect the steps together in the following specified order:

Configure the Text File Input step as follows:

After the name of the file has been typed in, click on the Add button; the file will be added to the selected files located beneath as follows:

Configure the Get Variable step as follows:

Configure the Filter step as follows:

Configure the Microsoft Excel Output step as follows:

Save the transformation with the suggested name read-customers.ktr.

The transformation design is now complete. You can now go to the Design a simple job (Simple) recipe and create the job that will use this transformation.

Create a new empty job. To do this, you can perform either of the following steps:

Go to the Job settings dialog and define a new parameter called p_country. Do this as follows:

Once the dialog opens, select the Parameters tab and add a new parameter called p_country.

Select the Design tab from the left-hand side view to display the list of tasks from their corresponding category folders. Look for the following tasks and drag-and-drop them into the working area on the right:

Connect the tasks together in the following specified order:

Configure the Delete File task as follows:

Configure the Transformation task as follows:

Under the Transformation specification tab, configure the name of the file for the transformation we're going to call. The transformation's file is located in the same samples directory where the job resides. In any case, a good approach is to type the filename in a way that is location independent, using a system variable to parameterize the directory name where the file is located. In our case, the complete filename is ${Internal.Job.Filename.Directory}/read-customers.ktr.

Save the job with the suggested name, export-job.kjb, in the same directory where you previously saved the transformation.

Try to execute the processes from the Spoon GUI. This is normally done by the developer to test or debug his/her jobs and transformations. To do this, click on the Run button, which can be identified by the green triangle icon located in the toolbar.

After the execution, if anything gets terminated successfully, the task icons will be decorated by a green mark that indicates that the specific tasks were executed successfully. Moreover, down in the working area, the Job Entry log details view indicates the results for every single task called by the job.

From the PDI home directory, open the kitchen.bat (or pan.bat) script.

Scan through the script's code until you find the following lines of code:

if "%PENTAHO_DI_JAVA_OPTIONS%"=="" set PENTAHO_DI_JAVA_OPTIONS=-Xmx512m

Change the value of the variable PENTAHO_DI_JAVA_OPTIONS to the required memory value for example, to 1024:

set PENTAHO_DI_JAVA_OPTIONS=-Xmx1024m

Save the file and exit.

From the PDI home directory, open the kitchen.sh script (or pan.sh).

Scan through the script's code until you find the following lines of code:

if [ -z "$JAVAMAXMEM" ]; then
  JAVAMAXMEM="512"
fi

if [ -z "$PENTAHO_DI_JAVA_OPTIONS" ]; then
  PENTAHO_DI_JAVA_OPTIONS="-Xmx${JAVAMAXMEM}m"
fi

Change the value of the variable JAVAXMEM to the required memory value for example, to 1024:

  JAVAMAXMEM="1024"

Save the file and exit.

Go to Control Panel and open the Environment variables dialog window.

Create a new system variable by clicking on the New button in the System Environment Variable section of the Environment variables dialog window.

Add a new variable called PENTAHO_DI_JAVA_OPTIONS.

Set the value of the variable. If we want to assign 1024 MB to PDI, for example, we set the variable's value to –Xmx1024m.

Click on OK to confirm and close the New System Variable dialog window.

Click on OK to close the System Environment Variables dialog window. This change will affect both scripts for jobs and transformations without any additional requirement.

Go to the User home directory and open the .bash_profile script file. If it does not exist, create a new one.

Add the following line to the script file:

export PENTAHO_DI_JAVA_OPTIONS="-Xmx1024m"

Save and close the file. Remember that this new environment variable will either be visible in the user context starting from the next login or after closing and re-opening your terminal window.

Open the command-line terminal and go to the <book_samples>/sample1 directory.

Let's start the sample job. To identify which job file needs to be started by Kitchen, we need to use the –file argument with the following syntax:

–file: <complete_filename_to_job_file>

Remember to specify either an absolute path or a relative path by properly setting the correct path to the file. The simplest way to start the job is with the following syntax:

$ kitchen.sh –file:./export-job.kjb

If you're not positioned locally in the directory where the job files are located, you must specify the complete path to the job file as follows:

$ kitchen.sh –file:/home/sramazzina/tmp/samples/export-job.kjb

Another option to start our job is to separately specify the name of the directory where the job file is located and then give the name of the job file. To do this, we need to use the –dir argument together with the –file argument. The –dir argument lets you specify the location of the job file directory using the following syntax:

–dir: <complete_path_to_ job_file_directory>

So, if we're located in the same directory where the job resides, to start the job, we can use the following new syntax:

$ kitchen.sh – dir:. –file:export-job.kjb

If we're starting the job from a different directory than the directory where the job resides, we can use the absolute path and the –dir argument to set the job's directory as follows:

$ kitchen.sh –dir:/home/sramazzina/tmp/samples –file:export-job.kjb

Normally, PDI manages input parameters for the executing job. To set parameters using the command-line script, we need to use a proper argument. We use the –param argument to specify the parameters for the job we are going to launch. The syntax is as follows:

-param: <parameter_name>= <parameter_value>

Our sample job and transformation does accept a sample parameter called p_country that specifies the name of the country we want to export the customers to a file. Let's suppose we are positioned in the same directory where the job file resides and we want to call our job to extract all the customers for the country U.S.A. In this case, we can call the Kitchen script using the following syntax:

$ kitchen.sh –param:p_country=USA -file=./export-job.kjb

Of course, you can apply the –param switch to all the other three cases we detailed previously.

In Windows, a PDI job from the filesystem can be started by following the same rules that we saw previously, using the same arguments in the same way. The only difference is in the way we specify the command-line arguments.

Any time we start the PDI jobs from Windows, we need to specify the arguments using the / character instead of the – character we used for Linux or Mac. Therefore, this means that:

-file: <complete_filename_to_job_file>

Will become:

/file: <complete_filename_to_job_file>

And:

–dir: <complete_path_to_ job_file_directory>

Will become:

/dir: <complete_path_to_ job_file_directory>

From the directory <book_samples>/sample1, if you want to start the job, you can run the Kitchen script using the following syntax:

C:\temp\samples>Kitchen.bat /file:./export-job.kjb

Regarding the use of PDI parameters in command-line arguments, the second important difference on Windows is that we need to substitute the = character in the parameter assignment syntax with the : character. Therefore, this means that:

–param: <parameter_name>= <parameter_value>

Will become:

/param: <parameter_name>: <parameter_value>

From the directory <book_samples>/sample1, if you want to extract all the customers for the country U. S. A, you can start the job using the following syntax:

C:\temp\samples>Kitchen.bat /param:p_country:USA /file:./export-job.kjb

The Pan script starts PDI transformations. On Linux or Mac, you can find the pan.sh script in the PDI home directory. Assuming that you are in the same directory, <book_samples>/sample1, where the transformation is located, you can start a simple transformation with a command in the following way:

$ pan.sh –file:./read-customers.ktr

If you want to start a transformation by specifying some parameters, you can use the following command:

$ pan.sh –param:p_country=USA –file:./read-customers.ktr

In Windows, you can use the Pan.bat script, and the sample commands will be as follows:

C:\temp\samples>Pan.bat /file:./read-customers.ktr

Again, if you want to start a transformation by specifying some parameters, you can use the following command:

C:\temp\samples>Pan.bat /param:p_country=USA /file:./read-customers.ktr

Open a command-line window and go to the <book_samples>/sample2 directory.

Kettle uses Apache VFS to let you access a set of files from inside an archive and use them directly in your processes; this is because it considers the archived content as a sort of virtual filesystem (VFS). To access a file in the archive, you need to give the complete path using a specific URI. In case you're going to access files contained in a .zip archive, the syntax to be used for the URI is as follows:

zip://arch-file-uri[!absolute-path]

On the other hand, if we wanted to access files contained in a .tar.gz file, we need to use the following syntax:

tgz://arch-file-uri[!absolute-path]

To identify which job file needs to be started, we need to use the –file argument with the following syntax; but now, the syntax has changed because we need to use, as a value, the URI with the syntax we saw in the preceding step:

–file: <complete_URI_to_job_file>

Remember that because we are talking about a URI to the file, we always need to consider the absolute path to the archive file followed by the path to the file we are going to start as a job.

To start the sample job named export-job1.kjb from within a .zip archive, use the following syntax:

$ kitchen.sh –file:'zip:///home/sramazzina/tmp/samples/samples.zip!export-job1.kjb'

The usage of parameters from within the command-line tool is the same as we saw in the Executing PDI jobs from a filesystem (Simple) recipe. So, if we want to extract all the customers from the country U. S. A, the command to use is as follows:

kitchen.sh -param:p_country=USA –file:'zip:///home/sramazzina/tmp/samples/samples.zip!export-job1.kjb'

Use the same command to start a job from within a .tar.gz archive, but with a different filesystem type in the URI. Here is a sample of a simple job that has been started without parameters:

$ kitchen.sh –file:'tgz:///home/sramazzina/tmp/samples/samples.tar.gz!export-job1.kjb'

The following syntax is for a simple job that has been started with parameters:

kitchen.sh -param:p_country=USA –file:'tgz:///home/sramazzina/tmp/samples/samples.tar.gz!export-job1.kjb'

Different from what we saw in the Executing PDI jobs from a filesystem (Simple) recipe, this time the –dir argument does not make any sense because we always need to access the archive through its complete URI and the –file argument.

Starting a PDI job from within an archive file on Windows requires for the same rules we saw previously to be followed using the same arguments in the same way.

Any time we start the PDI jobs from Windows, we need to specify the arguments using the / character instead of the – character we used in Linux or Mac. Therefore, this means that the –file argument will change from:

–file: <complete_URI_to_job_file>

To:

/file: <complete_URI_to_job_file>

Go to the directory <books_samples>/sample2; to start your sample job from within the ZIP archive, you can start the Kitchen script using the following syntax:

C:\temp\samples>Kitchen.bat /file:'zip:///home/sramazzina/tmp/samples/samples.zip!export-job1.kjb'

Let's create another example using parameters in the command. Go to the directory <books_samples>/sample2; to start the job by extracting all the customers for the country U. S. A, you can use the following syntax:

C:\temp\samples>Kitchen.bat /param:p_country:USA /file:'zip:///home/sramazzina/tmp/samples/samples.zip!export-job1.kjb'

PDI transformations are always started using Pan scripts. On Linux or Mac, you can find the pan.sh script in the PDI home directory. To start a simple transformation from within an archive file, go to the <book_samples>/sample2 directory and type the following command:

$ pan.sh –file: 'tgz:///home/sramazzina/tmp/samples/samples.tar.gz!export-job1.kjb'

Or, if you need to specify some parameters, type the following command:

$ pan.sh –param:p_country=USA –file:./read-customers.ktr

On Windows, you can use the Pan.bat script and the sample commands to start our transformation as follows:

C:\temp\samples>Pan.bat /file='zip:///home/sramazzina/tmp/samples/samples.zip!read-customers1.ktr'

Or, if you need to specify some parameters through the command line, type the following command:

C:\temp\samples>Pan.bat /param:p_country:USA /file='zip:///home/sramazzina/tmp/samples/samples.zip!read-customers1.ktr'

Let's start our sample job from the repository; to do this, we need to use two new arguments:

So, because the ID of our repository is sample3, we'll start the sample job with the following syntax:

$ kitchen.sh –rep:sample3 –job: export-job

Suppose that the job is not located in the root directory of our repository, but is stored in the pdi_book directory; to start it, we must specify the correct path to the job in the repository with the following syntax using the –dir argument:

$ kitchen.sh –rep:sample3 –dir:/pdi_book  -job:export-job

Parameter usage from within the command-line tool is exactly the same as we saw in the previous recipes. So, if we want to extract all the customers from the country U.S.A, a sample of the command to use is as follows:

kitchen.sh -param:p_country=USA ––rep:sample3 –job: export-job

Starting a PDI job from within a filesystem repository on Windows requires for the same rules we saw previously to be followed with the exception that, as usual, we need to specify the arguments using the / character instead of the – character that we used for Linux or Mac. So, the –rep argument will change to /rep:

/rep: < id_of_the_repository >

And the –job argument will change to /job:

/job: < name_of_the_job >

Let's suppose that the job is located in the root directory of our repository; we can call the Kitchen script using the following syntax:

C:\temp\samples>Kitchen.bat /rep:sample3 /job:export-job

About the use of PDI parameters through the command-line arguments: let's suppose we run our job by extracting all the customers for the country U.S.A; we can call our job using the following syntax:

C:\temp\samples>Kitchen.bat /rep:sample3 /job:export-job /param:p_country:USA

The process of starting a PDI job in a database repository is the same as in a filesystem repository. The only difference in this case is that a database repository is an authenticated source, so it requires a username and password.

The –user parameter lets you specify a valid username to connect to the repository. If you are on Linux/Mac, the syntax will be as follows:

–user: <repository_username>

For Windows, you need to change the – character to the / character; the syntax is as follows:

/user: <repository_username>

The –pass argument lets you specify a valid password for the user you are using to connect to the database repository; the syntax in this case for Linux/Mac is as follows:

–pass: <repository_user_password>

And for Windows, the syntax is as follows:

/pass: <repository_username>

Suppose that our job, called export-job.kjb, is located in a database repository instead of in a filesystem repository, and that we are connecting to the user pdiuser with the password password. The example command to start the job in Linux/Mac is as follows:

$ kitchen.sh –user:pdiuser –pass:password –rep:sample3 –dir:/pdi_book  -job:export-job

Whereas in Windows, the syntax is as follows:

C:\temp\samples>Kitchen.bat /user:pdiuser /pass:password /rep:sample3 /job:export-job

Open a command-line window and go to the <book_samples>/sample1 directory.

Any time that we are going to start a job or a transformation, we can manually set the verbosity of our log output. The more verbosity you choose to have, the more logs will be produced. To do this, we can use the –level argument specified for Linux/Mac as follows:

–level: <logging_level>

And for Windows, the argument specified is as follows:

/level: <logging_level>

The –level argument lets you specify the desired logging level by choosing its value from a set of seven possible values specified as follows:

To start the job in Linux/Mac with a log level set to the Error level, we can give the following command:

$ kitchen.sh –file:/home/sramazzina/tmp/samples/export-job.kjb –level:Error

To start the job in Windows with a log level set to the Error level, we can give the following command:

C:\temp\samples>Kitchen.bat /file C:\temp\samples\export-job.kjb /level:Error

The log produced by our Kettle processes is an invaluable resource to properly diagnose problems and solve them quickly. So, it is a good rule of thumb to save the logs and eventually archive them for future reference.

In case you are launching your jobs from the command line, there are different ways to save the log.

The first thing we can do is save the log to a file using the logfile argument. This argument lets you specify the complete path to the logfile name.

To set the logfile name on Linux/Mac, use the following syntax:

–logfile: <complete_logfilename>

To set the logfile name on Windows, use the following syntax:

/logfile: <complete_logfilename>

Let 's suppose that we are going to start the export-job.kjb Kettle job, and we want a Debug log level and to save the output to a specified logfile called pdilog_debug_output.log. To do this on Linux/Mac, type the following command:

$ kitchen.sh –file:/home/sramazzina/tmp/samples/export-job.kjb –level:Debug –logfile:./pdilog_debug_output.log

To set the logfile on Windows, type the following command:

C:\temp\samples>Kitchen.bat /file:C:\temp\samples\export-job.kjb /level:Debug /logfile:.\pdiloc_debug_output.log

As soon as your PDI starts, it will start filling a buffer that contains all the rows produced by your log. This is interesting because it lets you keep the memory usage under control. By default, PDI maintains the first 5000 logs' rows produced by your job in this buffer. This means that if your ETL process produces more than 5000 rows, the output log is truncated.

To change the length of the log buffer, you need to use the maxloglines argument; this argument lets you specify the maximum number of log lines.

To set the maximum number of log lines that are kept by Kettle on Linux/Mac, use the following argument:

–maxloglines: <number_of_log_lines_to_keep>

To set the maximum number of log lines that are kept by Kettle on Windows, use the following argument:

/maxloglines: <number_of_log_lines_to_keep>

If you specify 0 as the value for this argument, PDI will maintain all of the log lines produced.

Another method to limit the number of log lines kept internally by the PDI logging system is to filter log lines by age. The maxlogtimeout argument lets you specify the maximum age of a log line in minutes before it is removed by the log buffer.

To set the maximum age of a log line on Linux/Mac in minutes, use the following argument:

–maxlogtimeout: <age_of_a_logline_in_minutes>

To set the maximum age of a log line on Windows in minutes, use the following argument:

/maxlogtimeout: <age_of_a_logline_in_minutes>

If you specify 0 as the value for this argument, PDI will maintain all the log lines indefinitely.

Let's suppose, for example, that we're going to start the export-job.kjb Kettle job and that we want to keep 10000 rows in our log buffer. In this case, the command we need to use in Linux/Mac is as follows:

$ kitchen.sh –file:/home/sramazzina/tmp/samples/export-job.kjb –level:Debug –logfile:./pdilog_debug_output.log –maxloglines:10000

Sometimes we need to start a job or a transformation but we do not have the details of the repository we are going to interact with. The first thing we need to know is the name of the repository we are going to connect to to start our process. To get the name of the available repositories, we can use the listrep command-line argument.

The usage is very simple because it does not need any value, just the name of the argument specified in the command line.

Imagine that we need to find the list of the available repositories on Linux/Mac; the command to give is as follows:

$ kitchen.sh –listrep

To do the same thing on Windows, the command is written as follows:

C:\temp\samples>Kitchen.bat /listrep

The result we get is the repositories listed in a clear and concise form with the repository ID in the first column and the repository name in the second column:

INFO  19-03 23:18:51,675 - Kitchen - Start of run.
INFO  19-03 23:18:51,695 - RepositoriesMeta - Reading repositories XML file: /home/sramazzina/.kettle/repositories.xml
List of repositories:
#1 : sample3 [PDI Book Samples]  id=KettleFileRepository

The next step after we have found the repository we were looking for could be to look for a job or transformation located somewhere in the repository.

To do this, we need to get the list of available directories in the repository using the listdir argument used together with the following arguments:

To find the list of the available directories in the root of the repository rep3, the command to fire on Linux/Mac is as follows:

$ kitchen.sh –rep:sample3 –listdir

To find the list of the available directories in the root of the repository rep3, the command to fire on Windows is as follows:

C:\temp\samples>Kitchen.bat /rep:sample3 /listdir

The command returns the list of available directories in the following form:

INFO  20-03 07:07:17,236 - Kitchen - Start of run.
INFO  20-03 07:07:17,252 - RepositoriesMeta - Reading repositories XML file: /home/sramazzina/.kettle/repositories.xml
dir2
dir1

The directory dir1 has a subdirectory, subdir11; to show this directory, we need to specify another command that for Linux/Mac is as follows:

$ kitchen.sh –rep:sample3 –dir:dir1 –listdir

And for Windows, the command is as follows:

C:\temp\samples>Kitchen.bat /rep:sample3 /dir:dir1 /listdir

PDI will show us the children of the directory dir1 as follows:

INFO  20-03 07:07:34,324 - Kitchen - Start of run.
INFO  20-03 07:07:34,339 - RepositoriesMeta - Reading repositories XML file: /home/sramazzina/.kettle/repositories.xml
subdir11

If you're checking the directories of an authenticated repository, the command will change as follows for Linux/Mac:

$ kitchen.sh –user:pdiuser –pass:password –rep:sample3 –listdir

And the command will change as follows for Windows:

C:\temp\samples>Kitchen.bat /user:pdiuser /pass:password /rep:sample3 /listdir

The output of the command will remain the same.

Now that we know about the internals of our repository, we're ready to look for our jobs.

The argument used to show the list of the available jobs in a specified directory is listjob. This argument must be used together with the following:

To find the list of the available jobs in the root directory of the repository rep3, the command to fire on Linux/Mac is as follows:

$ kitchen.sh –rep:sample3 –listjobs

To find the list of the available jobs in the root directory of the repository rep3, the command to fire on Windows is as follows:

C:\temp\samples>Kitchen.bat /rep:sample3 /listjobs

The command returns the list of available jobs in the following form:

INFO  20-03 07:30:46,642 - Kitchen - Start of run.
INFO  20-03 07:30:46,657 - RepositoriesMeta - Reading repositories XML file: /home/sramazzina/.kettle/repositories.xml
export-job

If you're checking the jobs in an authenticated repository, the command on Linux/Mac will change in the following way:

$ kitchen.sh –user:pdiuser –pass:password –rep:sample3 –listjobs

If you're checking the jobs in an authenticated repository, the command on a Windows platform will change in the following way:

C:\temp\samples>Kitchen.bat /user:pdiuser /pass:password /rep:sample3 /listjobs

The Pan script lets us see the list of transformations contained in a directory of our repository.

To do this, we need to specify the –listtrans argument together with the same arguments specified for the –listjobs argument; for details about this, please refer to the previous paragraph to get a detailed explanation of the meaning and syntax of those arguments.

To find the list of the available transformations in the root directory of the repository rep3, the command to fire on Linux/Mac is as follows:

$ pan.sh –rep:sample3 –listtrans

To find the list of the available transformations in the root directory of the repository rep3, the command to fire on Windows is as follows:

C:\temp\samples>Pan.bat /rep:sample3 /listtrans

The command returns the list of available transformations in the following form:

INFO  20-03 07:35:10,073 - Pan - Start of run.
INFO  20-03 07:35:10,103 - RepositoriesMeta - Reading repositories XML file: /home/sramazzina/.kettle/repositories.xml
read-customers

Anything applied to the display of jobs contained in a specific directory and the ability to apply the same command to an authenticated repository applies here to transformations as well; the only recommendation is to remember to use the Pan script instead of the Kitchen script.

Dumping a job stored in a repository, either authenticated or not, is an easy thing. To try the following examples, use the filesystem repository we defined during the recipe Executing PDI jobs from the repository (Simple).

To export a job and all of its dependencies, we need to use the export argument followed by the base name of the .zip archive file that we want to create. Remember that you don't need to specify any extension for the exported file because it will get attached automatically.

This kind of repository is unauthenticated, but you can do exactly the same things using an authenticated repository by specifying the username and password to connect to the repository.

To export the job export-job from the samples3 repository on Linux/Mac, type the following command:

$ kitchen.sh -rep:sample3 -job:export-job -export:sample-export

To export the job export-job from the samples3 repository on Windows, type the following command:

C:\temp\samples>Kitchen.bat /rep:sample3 /job:export-job /export:sample-export

You can do the same things to export jobs stored in the plain filesystem; in this case, the job must specify the files related to the job that is to be exported:

For details about the usage of these arguments, see the recipe Executing PDI jobs from a filesystem (Simple). Let's go to the directory <books_samples>/sample1.

To try to export all of the files related to the export-job.kjb job in Windows, type the following command:

kitchen.sh -file:./export-job.kjb -export:sample-export

To try to export all of the files related to the export-job.kjb job in Linux/Mac, type the following command:

C:\temp\samples>Kitchen.bat /file:./export-job.kjb /export:sample-export

Every time you start a PDI process, either jobs or transformations, the script that started the job or the transformation gets back from PDI with a return code that gives an indication about whether the process terminated successfully or not.

In case the process terminated unsuccessfully, the code gives you an overall indication of what happened.

Looking on the Internet at the Pentaho wiki and at some blogs (an interesting article was published recently on this topic in Diethard Steiner's blog at http://diethardsteiner.blogspot.it/2013/03/pentaho-kettle-pdi-get-pan-and-kitchen.html), you can easily find a summary table that describes these error codes and their meaning, which we have described for reference purposes in the There's more… section.

To display this code on Linux/Mac, edit the kitchen.sh script and add the following command at the end of the PDI script. This command returns the exit code of the last called process:

exit?1

To display this code on a Windows platform, edit the Kitchen.bat script and add the following command at the end of the PDI script. This command returns the exit code of the last called process:

echo %ERRORLEVEL%

You can do the same with the Pan scripts. As soon as the script terminates, it displays the error code. You can try it out by adding it to your scripts and then calling one of your sample jobs or transformations. Getting this code is a very interesting thing because as soon as you call Kitchen or Pan scripts from inside another script, the caller is able to take action in case something goes wrong. This means that we can design an error handling strategy.

To schedule ETL processes on Linux, we always use the cron scheduler. The task to do this is very easy, while understanding the syntax of the cron schedules is a bit more complicated.

To add a new schedule, you need to edit the crontab file; to do this, type the following command from the Linux command line:

crontab –e

Depending on the distribution, you can also use some graphical tools to do the same, but as I always suggest, it is better to learn the command-line commands to give you complete freedom and portability.

To schedule a process, you need to specify the time at which the command will run. To do this, there is a particular syntax that lets you specify minutes, hours, the month date, and the month year. The format of any of these items is as follows:

You can follow some simple rules to specify ranges or multiple values for any of the preceding items, summarized as follows:

So, let's suppose that we want to schedule a job in the <book_samples>/sample1 directory called export-job.kjb to execute at every half an hour, at 20 and 50 minutes past the hour every day, except on weekends. To do this, you need to add a new schedule by typing the following command:

20,50 * * * 1-5 <pdi_home>/kitchen.sh –file <books_samples>/export-job.kjb

After you have typed the command, save and quit the crontab editor by pressing the Esc key; then type :wq, and then press the Enter key. The schedule will immediately be available. As you can see, in this case, you need to specify the complete path to the PDI scripts and to the export-job.kjb job. Of course, scheduling a Pan script is just a matter of changing the script name.

Further references about how to schedule tasks with the cron scheduler can be found on Wikipedia at http://en.wikipedia.org/wiki/Cron.

Go to Control Panel | Administrative Tools | Task Scheduler.

Select Create a Basic Task from the menu on the right.

The Create Basic Task Wizard dialog will be displayed. Enter the task name (for example, Example PDI job). Click on Next.

Select the schedule you want to apply. You can either select the schedule based on the calendar or on an event. Let us suppose our schedule is based on the calendar and we want our program to be scheduled daily. Select the Daily radio button and click on Next.

Insert the time you want the event to fire and its recurrence multiplicity (Recur every). We want, for example, the event to fire everyday at 4.00 A.M. Click on Next.

Select the action you want to accomplish; in our case, we select Start a program. Click on Next.

In the Program/script field, insert the full path and the name of the script you want to start (Kitchen.bat or Pan.bat). In the Add arguments field, insert the usual information you pass along to the Kitchen or Pan script to start your job or transformation properly. Click on Next.

At the end, a summary dialog will open showing you all the parameters you inserted to completely define your schedule.

Click on Finish.

Your job will now appear in the list of waiting jobs.

The at command works the same as cron, so it becomes very easy to schedule our job with Kitchen or our transformation with Pan.

Let's suppose that we want to schedule a job in the <book_samples>/sample1 directory called export-job.kjb to be executed everyday at 8:00 AM with the exception of the weekends. To do this, you need to add a new schedule by typing the following command:

at 8:00 /every:M,T,W,Th,F <pdi_home>\Kitchen.bat /file:<books_samples>\export-job.kjb

After you type the command, the schedule immediately becomes available. As you can see, in this case, you need to specify the complete path to the PDI scripts and to the export-job.kjb job.