The ways in which data can be set up, saved, accessed, and manipulated are extensive and varied. When setting up your data, choosing the format for your files is a process that requires applied thought. Some of the topics discussed here include data ingestion patterns, the data lake concept, and choosing a suitable file format.

Audience: Pentaho ETL developers and architects interested in learning data ingestion patterns, or anyone needing to bulk load data, change a schema, store intermediate data, or optimize query speed.

Pentaho MapReduce (PMR) allows ETL developers to design and execute transformations that run in Hadoop MapReduce. If one or more custom steps are used in the mapper, reducer, or combiner transformations, additional configuration will be needed to make sure all dependencies are available for use in the Hadoop environment.

Audience: Pentaho Data Integration (PDI) developers or administrators configuring PMR on a Hadoop cluster.

Pentaho MapReduce (PMR) allows ETL developers to design and execute transformations that run in Hadoop MapReduce. When PMR jobs are run, additional variables are injected into the Kettle variable space that can be used to enhance transformation that map, combine, or reduce. Also covers examples for using TaskID.

Audience: Pentaho Data Integration (PDI) developers or administrators configuring PMR on a Hadoop cluster.

This document is intended to provide insight and best practices for setting up Pentaho Data Integration (PDI) to work with MapR, and includes information about setting up and installing the MapR client tool that is required by PDI to run Pentaho MapReduce jobs. Also discusses setting up environments.

Certain configurations on the Hadoop ecosystem will be examined to make sure that the client is correctly setup before PDI will use it.

Audience: Cluster or server administrators, especially Hadoop administrators.

Pentaho Data Integration (PDI) includes multiple functions to push work to be done on the cluster using distributed processing and data locality acknowledgment. This document covers best practices to push ETL processes to Hadoop-based implementations. MapReduce and YARN are also covered here.

Audience: Cluster or server administrators, solution architects, or anyone with a background in big data processing. 

There are different techniques to process and parse Extensible Markup Language (XML) files stored in a Hadoop cluster. This best practice focuses on selecting and implementing the best strategy to parse XML for your use case.

Audience: Cluster or server administrators, solution architects, or anyone with a background in big data processing. 

This document covers some best practices on integrating R with PDI, including how to install and use R with PDI and why you would want to use this setup.

Audience: Data analysts, data scientists, and PDI users who need to use the variety of statistical and machine learning tools available in the R environment.

This document covers some best practices on integrating Pentaho software with Cloudera QuickStart VM, including how to configure the QuickStart VM so that Pentaho can connect to it.

Audience: Pentaho developers and system architects looking to experiment with PDI and Hadoop.

This is a collection of best practices on using Pentaho Analyzer with Impala data sources, including how to prepare and partition data and set configurations. You will also learn about schema recommendations and settings for Analyzer, Metadata, and the design and use of partitions.

Audience: Pentaho developers, system administrators, and architects.