This application demonstrates how to read from files by using the APIs
Before running this application:
- Ensure that you have access to a cluster running MapR-FS.
- Ensure that a text-based file that you have access to exists on the cluster. Note the path to the file.
- Decide on the number of bytes to read from the file.
To build and run it, download it from this page and copy it to a MapR client. Then, modify the
run.sh script in Building and Running C Applications on MapR-FS Clients to point to this sample application. Run the script and then run the application.
The application includes these header files:
The APIs are defined in
hdfs.h. The file
fcntl.h defines the file-access flags.
The application performs the actions that are described in the following sections.
Takes a filename and buffer size as input
After compiling the application, type the following command to launch the application and pass in the path and name of the file, as well as the size of the buffer to read data into:
hdfs_read <filename> <buffer size>
Sets an RPC timeout
hdfsSetRpcTimeout() is specific to the
libMapRClient version of
libhdfs and takes a value that is specified in seconds. The default is 99 seconds. If you change this value, set it either to 0 (which eliminates timeouts) or to a value greater than 30.
Connects to a filesystem, using an API that is supported in the hadoop-0.20.2 version of
The application tries to connect to the first MapR-FS cluster that is specified in the
mapr-clusters.conf file in the
MAPR_HOME/conf directory on the client. After connecting to the filesystem, the application returns a handle to the filesystem.
Stores the values of the arguments
The application stores them in a character array and in a variable of type
tSize. This datatype is defined in
hdfs.h and is a fixed-width, signed 32-byte integer type for storing the size of data for read or write operations.
Opens the file that you specified
The application opens the specified file, passing the following values to the
- The handle to the filesystem
- The name of the file, which you supplied when you launched the application.
- A flag to indicate the mode in which to open the file. In this case, the flag is
O_RDONLY, which specifies read-only mode.
- The default chunk size for the directory in which the file is either located or will be created. This value is specified by the 0 in the last parameter.
Although there are two other parameters in the
hdfsOpenFile() function – the fourth and fifth, the
libMapRClient version of
Creates a buffer of the size that you specified
This is the buffer that the application will read data into.
Reads an entire file with hdfsRead
The application calls the function
This function uses a WHILE loop. In each loop iteration, the function reads an amount of data that is equal to the size of the buffer. When the amount of bytes read is less than the size of the buffer, the end of the file has been reached and the function breaks the loop. The number of bytes read is added to a total in each iteration.
Seeks an offset and reads from that offset with
The application next calls the function
readAtOffset(), passing in 0 as the offset from which to start reading the file.
This function calls
hdfsSeek() to move to the specified offset in the file.
If the seek is successful, the function reads from that offset until the buffer is full. The function then returns the number of bytes that were read.
If the seek or the read is not successful (meaning
hdfsRead() returned -1), the function closes the file and returns -1.
The offset in the file is the next byte after the end of the data that was read.
Performs a positional read with
The application calls
positionalRead(), passing 100 as the offset from which to start the read.
The function reads data into the buffer, starting at offset 100, without first calling
hdfsSeek() to move the offset to that position. The offset is not moved to 100 before the read begins. The offset stays where it is, the read begins at offset 100, and (after the read) the offset remains where it was before the read. The offset in the file is ignored by the positional read.
Closes the file
Frees the buffer