Release of streamsx.hdfs toolkit Version 5.0.0

Release of streamsx.hdfs toolkit Version 5.0.0

What is new in version 5.0.0

• The streamsx.hdfs toolkit has been improved with a new operator HDFS2FileCopy.

  HDFS2FileCopy copies files in to directions:
  copyFromLocalFile : Copies a file from local disk to the HDFS file system.
  copyToLocalFile : Copies a file from HDFS file system to the local disk.

• It supports also two new parameters for all operators:
  credentials : credentials is a JSON string that contains key/value pairs for user , password and webhdfs .
  appConfigName : Streams application configuration that support credentials. And credentials is a JSON string that contains user, password and webhdfs.

• Adapting of streamsx.hdfs to support HDP 2.x and HDP 3.x file systems.

• Adapting of com.ibm.streamsx.hdfs.client.webhdfs java package to support knox HttpURLConnection.

• The pom.xml file has been upgraded to use Hadoop version 3.1.0 libraries.

streamx.hdfs.toolkits-4.4.1

• The pom.xml file has been updated.

• It is possible to run streamsx.hdfs without any installation of hadoop client and without environment variable HADOOP_HOME.

• The parameter configPath has to be set. The configPath is a directory that contains the hadoop configuration file (core-site.xml).

• It is recommended to copy the core-site.xml file in a directory in your project and set the parameter configPath.
  For example:

  cd  your-project
  mkdir etc
  cp ~/core-site.xml etc/
  

and in your SPL code:

 configPath : "etc"

streamx.hdfs.toolkits-4.4.0

streamx.hdfs.toolkits-4.4.0

What is new in version 4.4.0:
The maven pom.xml file has been upgraded to use hadoop 2.8.5 3rd party libraries:

streamx.hdfs.toolkits-4.3.0

The streamx.hdfs.toolkits-4.3.0 release delivers corrections for issue: #102

What is new in version 4.3.0
The operator HDFS2FileSink provides a new parameter tempFile.
The tempFile parameter specifies the name of the file that the operator writes to. When the file is closed the file is renamed to the final filename defined by the file parameter.

streamx.hdfs.toolkits-4.2.0

The streamx.hdfs.toolkits-4.2.0 release delivers corrections for FileNotFoundException issue #100 and issue #90
The release has been created on a RedHad 7.5 linux server with
IBM Streams Version=4.3.0.0 Build=20180725113900
This release fixed also divers deprecated hadoop functions.

IBM Streams HDFS Toolkit Version 4.1.0

The HDFS Toolkit has been improved to support the IBM Streams on Cloud.
A new pom.xml file download only the needed JAR libraries to build the toolkit.
The Hadoop JAR libraries are now integrated in the toolkit lib/ext directory.

The HDFS Toolkit has now a reconnection policy. It tries to reconnect to the HDFS when it fails
All operators have now 4 new parameters:
hdfsPassword
reconnectionPolicy
reconnectionBound
reconnectionInterval

It is now possible to submit an SPL application on IBM Streams Cloud without any
Hadoop client installation on Streams server.

A sample has been provided to describe the build and submit a SPL application with new
version of HDFS Toolkit.
https://github.com/IBMStreams/streamsx.hdfs/wiki/How-to-connect-to-TAE-vai-HDFS-Toolkit

streamsx.hdfs toolkit version 3.1.0

streamsx.hdfs toolkit version 3.1.0
The HDFS toolkit has been built wit Streams:
Version=4.2.1.1
ProductName=IBM Streams
Build=20170404211051
Architecture=x86_64
OS=Red Hat Enterprise Linux Server release 6.6 (Santiago)

HDFS Toolkit release for Streams 4.2

This is version 3.0.100 of the HDFS toolkit as released in Streams 4.2.

It includes:

• Support for BigInsights 4.2.
• Minor enhancements and fixes (issues #55 and #79)

HDFS for Bluemix Toolkit v3.5.1

This release adds support for the paid BigInsights for Apache Hadoop service on Bluemix.

HDFS for Bluemix Toolkit v3.5.0

This is an official release of the HDFS v3.5.0 toolkit to support the Analytics for Apache Hadoop service and the Streaming Analytics (Streams 4.1) service on Bluemix.

Highlights of the release include:

• Support for information governance