The stor data storage library
Marty Olevitch
Laboratory for Experimental Astrophysics
Physics Dept, Washington University in St. Louis
St. Louis, MO 63130 USA
marty-stor@cosray.wustl.edu
Contents
The stor data storage library is a collection of C routines to aid in the
storage of data acquired, especially in real-time situations such as saving
data from balloon flight or accelerator experiments. Use of this library
frees the programmer from dealing with the mechanics of saving the incoming
data. After initialization, one simple function call saves data. Other
calls can be used to find out how much data has been stored, etc.
Version 1.7 of the stor library has been run under UNIX (Solaris 2.5, 2.7,
Linux) using the GNU C compiler, under QNX using the Watcom compiler,
and MS-DOS using Borland C 3.1.
Hopefully, version 1.9 also works under these operating systems!
The diagram below shows a representation of the data directory tree. The
root of the tree is noted in the diagram by datadir. This directory
contains other directories, one for each ``run''. In the diagram, the runs
are 5-digit numbers. In general, a new run is started each time your
application program starts up. The root directory also contains a file
called .stor_cur_run (in the DOS version, this file is called
_cur_run) which contains the current run number. The library
maintains this file to keep track of the run numbers. Delete it only at the
risk of already saved data!
Inside each run directory are one or more data files. The names of these
files are represented as 6-digit numbers. The size of each data file is
specified by the programmer in the stor_init() function call. When this
limit is reached, the library will automatically open the next data file.
Typically, a data file will hold many events.
This code fragment illustrates the typical use of the library routines.
#include < stdio.h>
#include < stor.h>
...
unsigned char dp[DSIZE];
int count;
int ret;
...
/* initialize stor to save the data in the directory
* rooted at /data/misc/exp1. Each data file will be
* a maximum of 100,000 bytes */
ret = stor_init("/data/misc/exp1", 100000);
if (ret) {
fprintf(stderr,"Can't initialize stor (%s)\n",
stor_strerror());
exit(1);
}
...
while (1) {
/* read some data from the telemetry (not part of
* the stor library! */
count = read_telemetry_data(dp);
if (count > 0) {
/* save count bytes of data in the current data file */
ret = stor_save_data(dp, sizeof(unsigned char), count);
switch (ret) {
case 0:
/* okay */
break;
case 1:
/* okay and started new data file */
printf("started new file %d\n",
stor_get_cur_file());
break;
case -1:
/* error */
fprintf(stderr,"Error saving data (%s)\n",
stor_strerror());
exit(1);
break;
default:
fprintf(stderr,"Bad return code %d", ret);
break;
}
/* print some statistics */
printf("Total bytes saved: %d", stor_total_bytes());
printf("Bytes in current file: %d",
stor_bytes_cur_file());
}
}
The routines which may have problems opening files or writing data, etc all
return the value -1 as an error code. In most cases, the function
will also set an error message which may be retrieved using the
stor_strerror() function. For example, see
case -1: in the code fragment above.
You can get by using just the following three routines.
-
int stor_init(char *datadir, long maxfilesize)
- datadir is a pointer to a character string
containing the name of the
root data directory
- maxfilesize should contain the maximum possible file
size for each data file. If maxfilesize is zero, then
the file size is unlimited and new files will not be
automatically opened. New files can only be opened using
stor_next_file().
Initializes the stor module. This function must be run
before using any other function in the library. If all goes
well, it returns 0, else -1 if there is an
error.
-
int stor_save_data(unsigned char *dp, size_t size,
size_t nitems)
- dp is a pointer to the data to be saved
- size is the size in bytes of each data item
- nitems is the number of data items
Write the data pointed to by dp to the current data
file. Returns 0 if all went well, 1 if the amount of data
caused us to open a new data file, or -1 if there is an
error.
-
void stor_close(void)
-
Closes open files, frees allocated memory, resets flag
variables, etc. stor_init()
may be rerun afterwards to restart the stor package.
The following routines are not absolutely necessary, but can be useful if
you want to know some of the library's internal state.
-
long stor_get_cur_run(void)
-
Returns the integer value of the current run.
-
long stor_get_cur_file(void)
-
Returns the integer value of the current data file.
-
long stor_bytes_cur_file(void)
-
Returns the number of bytes written so far to the current
data file.
-
unsigned long stor_total_bytes(void)
-
Returns the number of bytes written since the last call to
stor_init.
-
char *stor_strerror(void)
-
Returns a pointer to the string containing the most recent
error message. The error message is updated whenever a
function returns -1. See
``Error handling'' above.
-
void stor_get_cur_run_str(char *s)
-
Fills in the character string pointed to by s with
the five-character name of the current run (for example
"00345").
-
void stor_get_cur_file_str(char *s)
-
Fills in the character string pointed to by s with
the six-character name of the current data file (for example
"001429").
-
void stor_cur_file_path(char *s)
-
Fills in the character string pointed to by s with
the full path name of the current data file
(for example "/data/tiger/00345/001429").
The following routines are not necessarily needed in ordinary use.
However, in special situtaions, they may be useful.
-
int stor_comment(char *s)
-
Writes the comment string pointed to by s to the
file stor_comments in the data directory, and also
the file stor_comments in the run directory. The
string has the date prepended. By default,
stor_init and
stor_close
insert a default comment. The default comment can be
suppressed by calling the
stor_no_default_comment
function.
-
int stor_flush(void)
-
Flushes any data to the current file.
Returns 0 if successful, else -1 if
there is an error.
-
long stor_next_file(void)
-
Closes the current data file and opens the next one.
Returns the new data file number (an non-negative integer)
if successful, else -1 if
there is an error.
-
int stor_next_run(void)
-
Closes the current run and starts the next one. The current
data file is closed and the new data file number is set to
0. Returns 0 if successful, else -1 if
there is an error.
-
void stor_no_default_comment(void)
-
Suppresses writing of a default comment by
stor_init and
stor_close.
This function should be run prior to stor_init.
See also
stor_comment.
-
void stor_version(char *s)
-
Writes the version to the string pointed to by s.