What is the use of node in data stage ?
What is the use of node in data stage ?
Actually data stage has a parallelism. You know in case of jobs, these should be optimized rather than maximized. So the number of nodes that are defined in configuration, is determined by the degree of parallelism of a data stage job.
With a larger number of nodes, a configuring file will generate larger number of all processes. So as compared to smaller number of nodes, it will turn processing overheads by a configuring file.
Therefore, you must be notified on the benefits of increased parallelism as compared to losses in processing efficiency while choosing the configuring file. It should be more ideal, if there is a small amount of data to be processed.
So configuration files with less number of nodes will be used and vice versa if larger amount of data is to be processed than the configuration of files with larger number of nodes.
One another aspect of data stage job design is, partitioning. Virtual settings jobs should be optimized, when high volume of data is processed in data stage jobs.
What is the use of node in data stage ?
The degree of parallelism, resources being used, and others in data stage are all determined during the run time based entirely on the configuration provided in the APT CONFIGURATION FILE.
Data stage jobs determine which node to run the process on, where to store the temporary data, where to store the dataset data, based on the entries provided in the configuration file. There is a default configuration file available whenever the server is installed.
You can typically find it under the “<>IBMInformationServerServerConfigurations” folder with the name “default.apt”. Bear in mind that you will have to optimize these configurations for your server based on your resources.
