Dar Documentation

LIBDAR

APPLICATION INTERFACE

TUTORIAL

for API version 5.7.x and above


Presentation

The Libdar library provides a complete abstraction layer for handling Dar archives. The general operations provided are:

  • archive creation,
  • file extraction,
  • archive listing,
  • archive testing,
  • archive comparison,
  • catalogue isolation,
  • archive merging,
  • dar_manager database manipulations
Note that Disk ARchive and libdar have been released under the Gnu General Public License (GPL). All code linked to libdar (statically or dynamically), must also be covered by the GPL. Commercial use is prohibited unless a contract has been agreeded with libdar's author.

This tutorial will show you how to use the libdar API.  Since release 2.0.0 the dar command-line executable also relies on this API, looking at it's code may provide a good illustration on the way to use libdar, the file src/dar_suite/dar.cpp is the primary consumer of the libdar API.

The sample codes provided here is solely illustrative and is not guaranteed to compile. More detailed API documentation is contained in the source code and can be compiled to the doc/html directory using Doxygen, which is also provided online.


Let's Start

Conventions

Language

Dar and libdar are written in C++, and so is the libdar API. While written in C++, libdar is easily usable by both C and C++ code. Access from other languages can be provided by specific bindings, like python bindings.

Header files

Only one include file is required in your program to have access to libdar:

#include <dar/libdar.hpp>

Libdar namespace

All libdar symbols are defined under the libdar namespace. You can either add the using namespace libdar; line at the beginning of your source files:

using namespace libdar;

get_version(....);

 or, as shown below, you can explicitly use the namespace in front of libdar symbols:


libdar::get_version(....);

Exceptions or no Exception

The library can be used with or without exceptions. For each example we will see a sample code for both ways. On the left you will find code relying on exceptions, on the right without exception:


example code using exceptions
example code not using exceptions

All exceptions used by libdar inherit from the pure virtual class Egeneric. The only method you will need to know about for any exception is the get_message() call, which returns a message string describing the message (in human language). The type of the error is defined by the class of the exception. The possible exception types follow:

class libdar::Egeneric
 the parent class of all exceptions (a pure virtual class)
class libdar::Ememory
 memory has been exhausted
class libdar::Ebug
 signals a bug, which is triggered when reaching some code that should never be executed
class libdar::Einfinint
 arithmetic error detected when operating on infinint
class libdar::Elimitint
 a limitint overflow is detected, indicating the maximum value of the limitint has been exceeded
class libdar::Erange
 signals a range error
class libdar::Edeci
 signals conversion problem between infinint and string (decimal representation)
class libdar::Efeature
 a requested feature is not (yet) implemented
class libdar::Ehardware
 hardware problem is found
class libdar::Euser_abort
 signals that the user has aborted the operation
class libdar::Ethread_cancel A program has requested the termination of the current thread while libdar was running
class libdar::Edata
 an error concerning the treated data has been encountered
class libdar::Escript
 the script executed between slices returned an error code
class libdar::Elibcall
 signals an error in the arguments given to a libdar call of the API
class libdar::Ecompilation
 a requested feature has not been activated at compilation time



 1 - First we *must* initialize libdar by checking the libdar version

  
    // we'll want to display some messages
#include <io.h>

    // we include this header to access lidbar API
#include <dar/libdar.h>

    // all sample code shown will be inside this
    // function for simplicity's sake
void my_sample_function()
{
   try
   {
      libdar::U_I maj, med, min;

        // first we MUST call get_version()

      libdar::get_version(maj, med, min);

      if(maj != libdar::LIBDAR_COMPILE_TIME_MAJOR ||
         med < libdar::LIBDAR_COMPILE_TIME_MEDIUM)
        throw libdar::Erange("initialization",
        "we are linking against a wrong libdar");
   }
   catch(libdar::Egeneric & e)
   {
      std::cout << e.get_message() << std::endl;
   }
}

  
  
    // we'll want to display some messages
#include <io.h>

    // we include this header to access lidbar API
#include <dar/libdar.h>

    // all sample code shown will be inside this
    // function for simplicity's sake
void my_sample_function()
{

   libdar::U_I maj, med, min;
   libdar::U_16 excode;
   std::string msg;

    // first we MUST call get_version()

   libdar::get_version_noexcept(maj, med, min,
                                excode, msg);

   if(excode != LIBDAR_NOEXCEPT)
   {
      std::cout << msg << endl;
      return;
   }
 
   if(maj != LIBDAR_COMPILE_TIME_MAJOR ||
      med < libdar::LIBDAR_COMPILE_TIME_MEDIUM)
   {
      std::cout << "we are linking against wrong libdar" << std::endl;
      return;
    }

The get_version() function must be called for several reasons :
  • you must check that the library you've dynamically linked with is compatible with the features you will be using. The major number must be the same, for no compatibility is assured between two libdar versions of different major numbers. While run-time compatibility is assured between medium numbers, the medium number must be greater or equal to the one used at compilation time to be sure that all the features you want are available in the libdar library you dynamically linked with. Changes between minor versions correspond to bug fixes and is not to imply any API change, thus no constraints is present there (just note the presence of more bugs in lower numbers).
  • the get_version() call, beside returning version information, does important initialization tasks for libdar. If not called first, the libdar library will not initialized properly and its behavior will be unpredictable. Note that you may call get_version() several time if you wish, the second time only the version information is returned, the libdar library is not reset or the like.
  • Last, if strong encryption support is activated at compilation time, libdar will by default performs libgcrypt initialization when get_version() is called if libgcrypt is not already initialized. You can avoid having dar initializing libgcrypt by calling get_version() with an additional argument set to "false" :
ger_version(maj, med, min, false)
  • Note that libgcrypt documentation indicates that libgcrypt must be initialized directly from the application not from an intermediate library, thus you should avoid having libdar initializing libgcrypt by using the get_version(maj, med, min, false) described just above and proceed from the application itslef to the initialization of libgcrypt *before* calling get_version(). This is particularily true in multi-thread environment with strong encryption activated in libdar at compilation time, if other part of your program (outside of libdar) also relies on libgcrypt.

2 - We should prepare the end right now!


As we saw, libdar used some datastructures (mutex, secured memory, etc.) that need to be released properly before ending the program. It is important to invoke the following function before exiting your program once you invoked get_version(). It is a good idea to implement this right now not to forget about it later:


libdar::close_and_clean()

In particular, closes_and_clean() makes the necessary for memory to be released in the proper order. Not calling close_and_clean() at the end of your program may result in uncaught exception message from libdar at the end of the execution. This depends on the compiler, libc and option activated in libdar at compilation time.

3 - Intercepting signals

libdar by itself does not make use of any signal (see signal(2) and kill(2)). However, gpgme library with which libdar may be linked with to support asymetrical strong encryption (i.e. encryption using public/private keys) may trigger the PIPE signal. Your application shall thus either ignore it (signal(SIGPIPE, SIG_IGN)) or provide an adhoc handle. By default the PIPE signal leads the receiving processus to terminate.


4 - Let's see the available features (Optional)

once we have called one of the get_version* function it is possible to access the list of features activated at compilation time:




void my_sample_function()
{
        // let's continue in the same function
bool ea = libdar::compile_time::ea();
bool largefile = libdar::compile_time::largefile();
bool nodump = libdar::compile_time::nodump();
bool special_alloc = libdar::compile_time::special_alloc();
U_I bits = libdar::compile_time::bits();
// bits is equal to zero for infinint,
// else it is equal to 32 or 64 depending on
// the compilation mode used.

bool thread = libdar::compile_time::thread_safe();
bool libz = libdar::compile_time::libz();
bool libbz2 = libdar::compile_time::libbz2();
bool liblzo = libdar::compile_time::liblzo();
bool libxz = libdar::compile_time::libxz();
bool libcrypto = libdar::compile_time::libgcrypt();
bool furtive_read = libdar::compile_time::furtive_read();

// to get the complete list of available routines check the file
// src/libdar/compile_time.hpp from source package *or*
// /usr/include/dar/compile_time.hpp from system header files

}

You can do what you want with the resulting values, like just to displaying the available libdar features (this is what dar -V does) or to terminating your program if you don't find a desired feature. However, verifying that features are available is not strictly necessary because libdar will tell you if an operation you call requires a feature that has not been activated at compilation time, by throwing an Ecompile exception (or returning the LIBDAR_ECOMPILATION error code if you are not using exceptions).


5 - User Interaction

a) The generic user_interaction class

To be able to send messages to and retreive information from the user, a special class called user_interaction is defined. Simply put, user_interaction is a virtual class from which you have to derive your own class in order to provide user interaction (a GUI's graphical interaction, for example). There is four methods among which only three have to be overridden:

void pause(const std::string &message);
this method is called by libdar when the library needs a yes/no answer to a question, which question is provided by the std::string message. The question given to pause() must be answered by returning normally (= "true") or throwing a Euser_abort exception if the user refused the proposition. Don't worry about throwing an exception in your code; it will be trapped by libdar if you don't want to manage exceptions, and are using libdar in the "no exception" method. But if you really don't want to throw exception from your code ignore this method (don't overwrite it in your class) but rather consider the following one instead:


bool pause2(const std::string &message);
This is an alternative method to pause() as seen above. In place of defining a pause() method in your inherited class, you can redefine the pause2() method. The only difference with pause() is that the user answer to the question is returned by a boolean value, your code does no more have to throw a Euser_abort exception to say "no". Note that you must not redefine both pause() and pause2()

void inherited_warning(const std::string &message);
libdar calls this protected method to display an informational message to the user. It is not always a warning as the name suggests, but sometimes just normal information.

std::string get_string(const std::string &message, bool echo);
This call is used to get an arbitrary answer from the user. This is mainly used to get a password from the user (when no password has been supplied for an encrypted archive), the echo argument indicates whether the user response should be displayed back on the screen. If echo is set to "false" the implementation of get_string() should hide the characters typed by the user (a password requested from the user).

 user_interaction * clone() const;
A copy operation must be implemented here. This is because libdar does not stores a reference to the user_interaction class it receives but rather keeps an  internal copy of it, so your class should also be ready to support a copy constructor and an assignement operator. A simple implementation of this method should be something like this (even if you don't want to use exceptions):

#include <new>
#include <dar/libdar.hpp>

class my_interaction : public libdar::user_interaction
{
public:
   bool pause2(const std::string & message);
   user_interaction *clone() const;
protected:
   void inherited_warning(const std::string & message);
};


user_interaction *my_interaction::clone() const
{
    user_interaction *ret = new (std::nothow) my_interaction(*this);
    if(ret == nullptr)
        throw Ememory("user_interaction_callback::clone");
    else
        return ret;
}

b) Or The callback interaction class

As an alternative to defining your own inherited class from libdar::user_interaction, libdar provides a class called user_interaction_callback which is an implementation of the user interaction, based on callback functions. This allows you to replace the three interactions methods (pause, warning and get_string) by three normal functions of your choice, which must be given to the user_interaction_callback's constructor. The clone() method is implemented internally, leaving only the three callback functions to be implemented. Look at dar's command line code for a practical example: dar's user interaction code is implemented using an instance of user_interaction_callback and three static functions in the module src/dar_suite/shell_interaction.cpp

Pay attention to the contextual value present in the arguments of these callback functions :

 
  // our own callback functions.
  // for the illustration of what these 'context' arguments
  // can be used for we will imagine the situation where
  // multiple windows or multiple threads may each one use
  // libdar, but all share the same callback functions.
 
typedef class t_window_type t_win;

  // this is an arbitrary type that here we will say
  // points to a graphical window object wrapped in a C++
  // class.
  // Note that the method show() wait_for_click() and so on
  // attributed to the t_win class are absolutely
  // imaginary. Any link to an existing class is a pure
  // coincidence...

void warning_callback(const std::string &x, void *context)
{
    (t_win *)(context)->show(x);
}
 
bool answer_callback(const std::string &x, void *context)
{
    click_type ret;

    (t_win *)(context)->show(x);
    ret = (t_win *)(context)->wait_for_click();

    return ret == click_OK;
}

std::string string_callback(const std::string &x, bool echo, void *context)
{
    (t_win *)(context)->show(x);
    if(!echo)
      (t_win *)(context)->set_hide_typed_char();
    (t_win *)(context)->wait_for_click();
    return (t_win *)(context)->read_text();
}

---------8<-------8<-------8<-------

  // So now each window can have its user_interaction object based on the same
  // user_interaction_callback object pointing to the same functions.
  // user_interaction_callback objects can be shared among different window objects

libdar::user_interaction_callback dialog =
        libdar::user_interaction_callback(&warning_callback, &answer_callback,
                                          &string_callback,
                                          (void *)get_current_windows_id());

  // just the "context" argument changes, and will be passed as is from the constructor to the callback
  // functions




6 - Let's create a simple archive

Now that we have seen user_interaction and exceptions let's start the real thing:

All the operations on archives are handled by the archive class which is defined in libdar/archive.hpp (no need to include this file it is done from <dar/libdar.hpp>). Each operation requires some mandatory parameters and some optional parameters. Optional parameters are gathered in an archive_option auxilliary class, which default constructor set them to their default values. We will see a bit further how to set these options, but for now let's keep the things simple:

  // creating an archive is simple; it is just
  // a matter of calling the "create" constructor
  // of the archive class. It may be used for full or
  // differential archives. We'll see an example for
  // a differential backup later.

  // first we need a user_interaction object, here
  // we will use three callback functions to set it up

libdar::user_interaction_callback dialog = libdar::user_interaction_callback(ptr1, ptr2, ptr3);
  // where ptr1, ptr2 and ptr3 are three callback
  // functions.

libdar::archive *my_arch =
     new libdar::archive(dialog,
        "/home",// saving all under this directory
        "/tmp", // where the slices will go
        "my_archive", // the basename of the slices
        "dar",  // the extension used for slices
        archive_options_create(), // default options
        nullptr);  // we will see later this parameter
                // in the progressive_report section
                // for now let ignore it by using nullptr

    // creating an archive is simple; it is just
  // a matter of calling the "create" constructor
  // of the archive class. It may be used for full or
  // differential archives. We'll see an example of
  // of differential archives later.

  // first we need a user_interaction object, here
  // we will use three callback functions to set it up
 
libdar::user_interaction_callback dialog = libdar::user_interaction_callback(ptr1, ptr2, ptr3);
  // where ptr1, ptr2 and ptr3 are three callback
  // functions.

U_16 exception,
std::string except_msg;

libdar::archive *my_arch =    
     new libdar::create_archive_noexcept(dialog, 
     "/home",  // saving all under this "root"
     "/tmp",   // where the slices will go
     "my_archive", 
            // the basename of the slices
     "dar", // dar's slice extensions
     archive_options_create(), // default options
     nullptr, // see the progressive_report section below
     exception, // this gives the status of the call
     except_msg); // and in case of error the cause.

if(exception != LIBDAR_NOEXCEPT)
  std::cout << "an error occurred: " << except_msg
            << std::endl;

When creating an archive, the just created object once the backup has complete can be used only as reference for an isolation or for a differential backups. You cannot use it for restoration, listing, or comparison, because the underlying file descriptors are opened in write only mode. An implementation which uses file descriptors in read-write access is not possible and is not a good idea anyway. Why? Because, for example, if you want to test the newly created archive, using the newly created object would make the test rely on information stored in virtual memory (the archive contents, the data location of a file, etc.), not on the file archive itself. If some corruption occurred in the file you would not notice it.

So to totally complete the cycle as we do not perform archive isolation or differential backup here, we must destroy the archive object we have just created, which will also close any file descriptors used by the object:


     delete my_arch;
libdar::close_archive_noexcept(my_arch, exception,
                        except_msg);

if(exception != LIBDAR_NOEXCEPT)
  std::cout << "an error occurred: " << except_msg
            << std::endl;

Here we have been using pointers to simplify the way to describe object terminason. Of course, it is also possible to use plain variable in place of pointers. The archive object is destructed at end of the block:

{
libdar::archive my_arch = libdar::archive(...);
}



Optional Arguments:


Back on the optional arguments. The archive constructor used above to create an archive uses an argument of type libdar::archive_option_create. In the above example, we called the constructor of this class directly withing the argument list of the class libdar::archive's constructor. This has for effect to built an anonymous temporary object of this class. Such a "just born" object has all the necessary options set to their default values. If you want to use non default options like compression, slicing, encryption, file filtering and so on, you must change the options to your need thanks to the appropriate method provided by the libdar::archive_options_create class. Assuming we want to make a compressed an sliced archive we would use the following code:


 
   // we define an options object to be able to use
   // non default options:
libdar::archive_options_create options;

  // so now we can modify it:
options.set_slicing(1024); // 1024 *bytes* per slice, well that's small
options.set_compression(bzip2); // default is no compression
options.set_compression_level(6); // default is 9

  // "dialog" below is the user_interaction object
 // we created earlier

libdar::archive *my_arch =
     new libdar::archive(dialog,
     "/home",  // saving all under this "root"
     "/tmp",   // where the slices will go
     "my_archive", // the basename of the slices
     "dar",  // the extension used for slices
     options, // the object we modified above
     nullptr);

   // we define an options object to be able to use
   // non default options:
libdar::archive_options_create options;

  // so now we can modify it:
options.set_slicing(1024); // 1024 *bytes* per slice, well that's small
options.set_compression(bzip2);  // default is no compression
options.set_compression_level(6); // default is 9

  // "dialog" below is the user_interaction object
 // we created earlier

U_16 exception,
std::string except_msg;

libdar::archive *my_arch =    
     libdar::create_archive_noexcept(dialog, 
     "/home",  // saving all under this "root"
     "/tmp",   // where the slices will go
     "my_archive", // the basename of the slices
     "dar", // dar's slice extensions
     options, // the object we modified above
     nullptr, 
     exception, // this gives the status of the call
     except_msg); // and in case of error the cause.

if(exception != LIBDAR_NOEXCEPT)
  std::cout << "an error occurred: " << except_msg
            << std::endl;


In the same way, each other operation (diff, testing, extraction, merging, ...) has a specific class that gathers the optional parameters. These classes are defined in the file dar/archive_options.hpp you are welcome to refer to for a complete up to date list of available fields. The advantage of this class is to not break ascendant compatibility of the API when new features get added, while it also improve readability of the code. This way, the major current number '5' of the API should stay for a longer time than previous numbers, because each new feature implementation broke the ascendant compatibility by adding an new argument to API calls.

7 - Testing the archive we have created


So, as explained previously, we must create a new archive object but this time using the "read" constructor, which difers from the "create" constructor we used earlier by the nature of its arguments:

  // "dialog" below is the user_interaction object
 // we created earlier

 my_arch = new libdar::archive(dialog,     
        "/tmp",  // where is the archive
        "my_archive", // slice name
        "dar",   // dar's archive extensions
        archive_options_read()); // default options
 		  // "dialog" below is the user_interaction object
 // we created earlier

 my_arch = libdar::open_archive_noexcept(dialog,     
        "/tmp",  // where is the archive
        "my_archive", // slice name
        "dar",   // dar's archive extensions
       archive_options_read(), // default options
       exception,// this gives the status of the call
       except_msg); // and in case of error the
                    // cause of the error

if(exception != LIBDAR_NOEXCEPT)
  std::cout << "an error occurred: " << except_msg
            << std::endl;


Now that we have opened the archive we can perform any operation on it. Let's thus start by testing the archive coherence:


   // for the exercice, we will change the default options:
archive_options_test options;

options.clear(); // this sets back all options to their
 // default values, this is for demonstration only as
 // here it was not necessary because the object had
 // just been created.
options.set_info_details(true); // to have a verbose output

ret = my_arch->op_test(dialog,
             options; // the non default options set above
             nullptr);  // we don't want a progressive report
   // for the exercice, we will change the default options:
archive_options_test options;

options.clear(); // this set back all options to default
 // here this is not required as the object has just bee
 // created, however it is used here for illustration that
 // you can recycle an archive_option_* object.
options.set_info_details(true); // to have a verbose output

ret = libdar::op_test_noexcept(dialog,
       my_arch,         // the archive to test
       options, // the non default options set above
       nullptr,  // we don't want a progressive report
        exception,// this gives the status of the call
       except_msg); // and in case of error the
                    // cause of the error

if(exception != LIBDAR_NOEXCEPT)
  std::cout << "an error occurred: " << except_msg
            << std::endl;


8 - listing archive contents


a) The simple way:

  // "dialog" below is the user_interaction object
 // we created earlier

 my_arch->op_listing(dialog,
             archive_option_listing()); // default options

   // "dialog" below is the user_interaction object
 // we created earlier
 
libdar::op_test_listing(dialog,
       archive_options_listing(), // default options
       exception,// this gives the status of the call
       except_msg); // and in case of error the
                    // cause of the error

if(exception != LIBDAR_NOEXCEPT)
  std::cout << "an error occurred: " << except_msg
            << endl;

Using this method, the library will performs the listing by calling the warning() method of the dialog object once for each file listed. The warning text will consist of a string for each file with the relevant information concatenated that would need to be parsed if individual information was desired. This may not be appropriate for your need and as such there is another way to get listing information:

b) The flexible way:


This new method gives information per directory libdar::archive::get_children_in_table(const std::string & dir) so you have to specify the directory you want to get content from. Second this method will return a list of object each one corresponding to an inode contained in that directory. Objects are of type libdar::list_entry

   // first we define the list that will receive information about the directory
std::vector<libdar::list_entry> info;

   // now we can ask the archive object for directory content,
   // the empty string corresponds to the root within
   // the archive:
info = my_arch->get_children_in_table("");


Now we have to exploit the table info. The class libdar::list_entry provides a set of method to retreive information about the directory entry:
  • the filename
  • inode type
  • whether the entry is a directory
  • whether the entry is a plain file
  • whether the entry is hard linked
  • whether the entry has EA
  • uid
  • gid
  • permission
  • last access date
  • last modification date
  • last change date
  • file size
  • compression ratio
  • whether the file is stored as a sparse file
  • whether the file is dirty (was modified at the time it was read for backup)
  • slice location
This is not an exhaustive list of method available for libdar::list_entry, more methods will come in future version, but existing ones should not change. Please refer to the file <dar/list_entry.hpp> for more information, or check the documentation about the class libdar::list_entry.

Here follows an simple example of use:

std::vector<libdar::list_entry> info;
std::vector<libdar::list_entry> tmp;
   // now we can ask the archive object for the content
   // the empty string corresponds to the root within
   // the archive:
info = my_arch->get_children_in_table("");

std::vector<libdar::list_entry>::iterator it = info.begin();

while(it != info.end())
{
   std::cout << it->get_name();
   if(it->is_dir())
   {
      tmp = my_arch->get_children_in_table(it->get_name());
      std::cout << "directory with " << tmp.size() << " entries" << std::endl;
   }
   else
      std::cout << std::endl;

   ++it;
}


9 - comparing with filesystem

We can compare file in an archive with the filesystem by calling the op_diff method of the class archive.


     ret = my_arch->op_diff(dialog,
                 "/home", // what directory to take
                          // as root we shall
                          // compare the archive
                          // contents to
                 archive_options_diff(), // default options
                 nullptr);   // we don't use progessive report



           
  ret = libdar::op_diff_noexcept(dialog,
                 my_arch, // the archive to use
                 "/home", // what directory to take
                          // as root we shall
                          // compare the archive
                          // contents to
                 archive_options_diff(), // default options
                 nullptr,    // we don't use progressive report
                  filesexception, // this gives the
                          // status of the call
                 except_msg); // and in case of
                          // error the cause of the
                          // error

if(exception != LIBDAR_NOEXCEPT)
  std::cout << "an error occurred: " << except_msg
            << std::endl;

Simple, no?

Just a note about the set_what_to_check() method argument of the libdar::archive_options_diff class. It may take several values:
  • libdar::cf_inode_type (default value): a file is considered as changed if its inode type has changed (directory/plain file/symbolic link/ ...)
  • libdar::cf_mtime : permission change is ignored, as well as ownership change
  • libdar::cf_ignore_owner : ownership change is ignored
  • lbdar::cf_all : all fields denoting a content's file change triggers a file changed status (ownership, permission, dates, inode type).

10 - restoring files

Restoration of files is done by calling the  op_extract method of class libdar::archive.


ret = my_arch->op_extract(dialog,
             "/tmp",   // where to restore files to
             archive_options_extract(), // default options
             nullptr);// no progressive report used
 		 
ret = libdar::op_extract_noexcept(dialog,
       my_arch,         // the archive to test
       "/tmp",   // where to restore files to
       archive_options_extract(), // default options
       exception,// this gives the status of the call
       except_msg); // and in case of error the
                    // cause of the error

if(exception != LIBDAR_NOEXCEPT)
  std::cout << "an error occurred: " << except_msg
            << std::endl;

Here as we used default options, we restore all the files stored in the archive into the directory /tmp (we also restore there the directory structure stored in the archive), but we could also make a flat restoration (ignore directory structure), as well as restore only some of the files. By default too, dar asks user confirmation before overwriting a file. You can change these options and many others using the methods the class libdar::archive_options_extract. We will here restore all that is under usr/lib and only files which filename ends with ".a", we want libdar to skip file that would lead to overwriting an existing file and also have libdar display files that have been skipped from the restoration. Note that a specific paragraph is designated to masks further in this tutorial, so don't panic :-)

archive_options_extract options;

options.set_selection(simple_mask("*.a", true));
options.set_subtree(simple_path_mask("usr/lib", true));
options.set_allow_over(false);
options.set_display_skipped(true);

ret = my_arch->op_extract(dialog,
             "/tmp",   // where to restore files to
             options, // non default options set just above
             nullptr);// no progressive report used
 		archive_options_extract options;

options.set_selection(simple_mask("*.a", true));
options.set_subtree(simple_path_mask("usr/lib", true));
options.set_allow_over(false);
options.set_display_skipped(true);

 ret = libdar::op_extract_noexcept(dialog,
       my_arch,         // the archive to test
       "/tmp",   // where to restore files to
       options, // non default options set just above
       nullptr,    // no progressive report used
       exception,// this gives the status of the call
       except_msg); // and in case of error the
                    // cause of the error

Last point about optional parameter concerns the set_what_to_check method. It serves two roles here:
  1. Which field are to be ignored when looking whether a file is more recent than the one in the filesystem (if this feature is enabled)
  2. Which field to avoid restoring (for example, when not having root privileges, avoid restoring ownership may be interesting instead of having a plethora of failure to restore ownership messages).


11 - Isolating the Catalogue

OK, I know, catalogue is not an English word (one would rather write catalog), but that's the name of the C++ class used in libdar, so we will keep using it here. Note that you don't have to directly access this class (if you really don't like French).

Isolating the catalogue creates a new archive that only contains the list of files and their attributes (ownership, dates, size, etc.), but no data, no EA and no FSA are stored in it. It is very similar to the same archive one gets if one makes a differential backup of a filesystem that has not changed since the creation of a reference archive. Archive isolation usage is very similar to the archive creation, but it uses a different constructor that has less arguments. Also we have optional arguments contained in class libdar::archive_options_isolate:

archive_options_isolate options;

  // we just want to have a compressed isolated catalogue
options.set_compression(gzip);

libdar::archive *my_cat = new libdar::archive(dialog,
                "/tmp",  // where the extracted
                         // catalogue is saved
                my_arch, // the archive of reference
                         // is the one we have been
                         // playing with previously
                "my_catalogue", // slice name
                "dar",   // file extension
                options) // non default options set above
                
      

archive_options_isolate options;

  // we just want to have a compressed isolated catalogue
options.set_compression(gzip);
          
libdar::archive *my_cat =
    libdar::op_isolate_noexcept(dialog,
               "/tmp",   // where is saved the
                         // extracted catalogue
                my_arch, // the archive of reference
                         // is the one we have been
                         // playing with previously
                "my_catalogue", // slice name
                "dar",   // file extension
                options, // non default options set above
                exception,
                       // this gives the status
                       // of the call
                except_msg);
                       // and in case of error the
                       // cause of the error

if(exception != LIBDAR_NOEXCEPT)
  std::cout << "an error occurred: " << except_msg
            << std::endl;

Now we have two archive objects. my_arch is a read-only object created by the "read" constructor in a previous paragraph of this tutorial. You can do any operations with it, like file restoration, file comparison, archive testing, as we have done in the previous sections. The second archive object is my_cat which is a write only object. It can only be used as a reference for another backup (a differential backup) or as a reference for a subsequent catalogue isolation (which would just clone the already isolated catalogue object here).

Note that once closed (i.e.: object has been destructed) you can re-open the isolated catalogue using the "read" constructor and use it as a read-only object (you can then test its integrity as seen previously).

So for now we will just destroy the extracted catalogue object, so that all its file descriptors get closed:


delete my_cat;   


close_archive_noexcept (my_cat, exception,
                        except_msg);

if(exception != LIBDAR_NOEXCEPT)
  std::cout << "an error occurred: " << except_msg
            << std::endl;


and we keep the my_arch object for our last operation:

12 - creating a differential backup

This operation is the same as the first one we did (archive creation). We will just provide the archive of reference as an optional parameter. If we had not destroyed my_cat above, we could have also used it in place of my_arch for exactly the same result:

archive_options_create options;

   // we provide here the reference to an
   // existing archive object, this implies that
   // the archive will be a differential backup
options.set_reference(my_arch);

   // as we are now used to options, we will set a more
   // complex set of other optional fields:
options.set_selection(not_mask(simple_mask("*~")));
options.set_empty_dir(true);
options.set_compression(bzip2);
options.set_compr_mask(not_mask(simple_mask("*.bz2")));
options.set_cache_directory_tagging(true);
options.set_slice_permission("0600");
options.set_slice_user_ownership("root");
options.set_slice_group_ownership("bin");
options.set_crypto_algo(crypto_blowfish);
  // if not specified with set_crypto_pass() the password
  // will be asked interactively to the user
options.set_slicing(100000000, 20480);
 
libdar::archive *my_other_arch =
     new libdar::archive(dialog,
     "/home",  // saving all under this "root"
     "/tmp",   // where the slices will go
     "my_archive", // the basename of the slices
     "dar", // dar's slice extensions
     options, // the optional parameter as defined above
     nullptr); // no progressive report
 archive_options_create options;

   // we provide here the reference to an
   // existing archive object, this implies that
   // the archive will be a differential backup
options.set_reference(my_arch);

   // as we are now used to options, we will set a more
   // complex set of other optional fields:
options.set_selection(not_mask(simple_mask("*~")));
options.set_empty_dir(true);
options.set_compression(bzip2);
options.set_compr_mask(not_mask(simple_mask("*.bz2")));
options.set_cache_directory_tagging(true);
options.set_slice_permission("0600");
options.set_slice_user_ownership("root");
options.set_slice_group_ownership("bin");
options.set_crypto_algo(crypto_blowfish);
  // if not specified with set_crypto_pass() the password
  // will be asked interactively to the user
options.set_slicing(100000000, 20480);

 libdar::archive *my_other_arch =    
     libdar::create_archive_noexcept(dialog, 
     "/home",  // saving all under this "root"
     "/tmp",   // where the slices will go
     "my_archive", // the basename of the slices
     "dar", // dar's slice extensions
     options, // the optional parameter as defined above
     exception, // thisgives the status of the call
     except_msg); // and in case of error the cause.

if(exception != LIBDAR_NOEXCEPT)
  std::cout << "an error occurred: " << except_msg
            << std::endl;

As previously, my_other_arch is a write only object that we won't need anymore. So we destroy it:


     delete my_other_arch;
libdar::close_archive_noexcept(my_other_arch,
                        exception,
                        except_msg);

if(exception != LIBDAR_NOEXCEPT)
  std::cout << "an error occurred: " << except_msg
            << std::endl;


We are at the end of this first part of the tutorial, where we have seen the general way to manipulate dar archives like dar command-line does. But we still have an object we need to destroy to cleanly release the memory used:


     delete my_arch;
libdar::close_archive_noexcept(my_arch, exception,
                        except_msg);

if(exception != LIBDAR_NOEXCEPT)
  std::cout << "an error occurred: " << except_msg
            << std::endl;


13 - Masks (Optional)

Mask are used to define which files will be considered and which will not. Libdar implements masks as several classes that all inherit from a virtual class that defines the way masks are used. This root class is the class mask and provides the mask::is_covered() method which libdar uses to determine which files are considered for the operation. There is several different basic masks classes you can use to build fairly complex masks, while it is possible you should not have to define you own mask classes, if the need arises, please contact libdar developer if you thing an additional class should take place beside the following ones:

class libdar::bool_mask
 boolean mask, either always true or false, it matches either all files or no files at all
class libdar::simple_mask
 matches as done by the shell on the command lines (see "man 7 glob")
class libdar::regular_mask
 matches regular expressions (see "man 7 regex")
class libdar::not_mask
 negation of another mask
class libdar::et_mask
 makes an *AND* operator between two or more masks
class libdar::ou_mask
 makes the *OR* operator between  two or more masks
class lbdar::simple_path_mask

matches if it is subdirectory of mask or is a directory that contains the specified path itself
class libdar::same_path_mask
 matches if the string is exactly the given mask (no wild card expression)
class libdar::exclude_dir_mask
 matches if string is the given string or a sub directory of it
class libdar::mask_list
 matches a list of files defined in a given file

Let's play with some masks :


      // all files will be elected by this mask
  libdar::bool_mask m1 = true;   

      // all file that match the glob expession "A*~" will match.
      // the second argument of the constructor tell whether the match is case sensitive so here
      // any file beginning by 'A' or by 'a' and ending by '~' will be selected by this mask:
  libdar::simple_mask m2 = libdar::simple_mask(std::string("A*~"), false);

      // m3 is the negation if m2. This mask will thus match
      // any file that does not begin by 'A' or 'a' and finishing by '~'
  libdar::not_mask m3 = m2;

      // this mask matches any file that is a subdirectory of "/home/joe"
      // and any directory that contains /home/joe, meaning
      // "/", "/home", "/jome/joe" and any subdirectory are matched.
      // here, the second argument is also case sensitivity (so
      //  "/HoMe" will not be selected by this mask as we set it to "true" here.
  libdar::simple_path_mask m4 = simple_path_mask("/home/joe", true);

      // now let's do some more complex things:
      // m5 will now match only files that are selected by both m2 AND m4
  libdar::et_mask m5;
  m5.add_mask(m2);
  m5.add_mask(m4);
     
      // we can make more interesting things like this, where m5 will select files
      // that match m2 AND m4 AND m3. But m3 = not m2 so now m5 will never
      // match any file...
  m5.add_mask(m3);

      // but we could do the same with an "ou_mask" and would get a silly
      // counterpart of m1 (a mask that matches any files)
  libdar::ou_mask m6;
  m6.add_mask(m2);
  m6.add_mask(m4);
  m6.add_mask(m3);

      // lastly, the NOT, AND and OR operation can be used recursively.
      // Frankly, it's possible to have masks reference each other!
  libdar::not_mask m7 = m6;
  m6.add_mask(m7);

The idea here is not to create object manually, but to link their creation to the action and choices the user makes from the user interface (Graphical User Interface of your aplication, for example)

Now that you've seen the power of these masks, you should know that in libdar masks are used at several places:
  • A first place is to select files against their names (without path information) this the argument of the set_selection() method of libdar::archive_options_* classes. The mask here does not apply to directories.
  • A second place is to select files against their path+name and it applies here to all type of files including directories, this is the argument of the set_subtree() method of libdar::archive_options_* classes. So with it you can prune directories, or in any other way restrict the operation to a particular subdirectory, as well as to a particular plain file for example. Important note about this second mask: what your own mask will be compared to by libdar is the absolute path of the file under consideration. If you want to exclude /usr/local/bin from the operation whatever is the fs_root value (which correspond the -R option of dar) using here a libdar::simple_mask("/usr/local/bin") as argument of libdar::archive_options_*::get_subtree() will do the trick.
An exception is the archive testing operation, which has no fs_root argument (because the operation is not relative to an existing filesystem), however the subtree argument exist to receive a mask for comparing the path of file to include or exclude from the testing operation. In this case the situation is as if the fs_root was set to the value "<ROOT>". For example, masks will be compared to "<ROOT>/some/file" when performing an archive test operation.
  • A third place concerns Extended Attributes (EA), this is the argument of the set_ea_mask() method of archive_options classes. It is applied to the full EA name in the form <domain>.<name> where <domain> is any string value like but not limited to the usual "user" or "system" domains.
  • A fourth place concerne the file to compress or to avoid compressing. This is the argument of the set_compr_mask() method of libdar::archive_options_* classes. it is works the same as set_selection() seen above, based only to filename without any path consideration.
  • A fifth place concern files that have need be prepared for backup, this is the argument of the set_backup_hook() method of libdar::archive_option_create class. I has to be used the same as set_subtree(). For more about this feature see the backup-hook feature in dar man page (-<, -> and -= options).



14 - Progressive Report (Optional)


Several method we saw previously of the class archive have an argument called progressive_report of type libdar::statistics* we have so far set to nullptr. We will see here how to use this feature.

The libdar::statistics pointed to object i(if progressive_report is not given nullptr) is regularily modified by the libdar::archive object during an operation (archive creation, archive testing and so on) and can also be accessed by another thread at the same time so this other thread can display the current status of the libdar operation, by mean of number of file treated, skipped, excluded, and so on.

libdar::statistics class is defined in <dar/statistics.hpp> header file and provides several methods to consult the different fields of the object:
       
        infinint get_treated() const;     ///< returns the current value of the treated counter
        infinint get_hard_links() const;  ///< returns the current value of the hard_links counter
        infinint get_skipped() const;     ///< returns the current value of the skipped counter
        infinint get_ignored() const;     ///< returns the current value of the ignored counter
        infinint get_tooold() const;      ///< returns the current value of the tooold counter
        infinint get_errored() const;     ///< returns the current value of the errored counter
        infinint get_deleted() const;     ///< returns the current value of the deleted counter
        infinint get_ea_treated() ;       ///< returns the current value of the ea_treated cou
        infinint get_byte_amount() const; ///< returns the current value of the byte_amount counter

As you see, the value returned by this methods is of type libdar::infinint (libdar's integer), whatever is the libdar flavor (infinint, 32bits or 64 bits). In order to display decimal representation of this type of variable you can use the class libdar::deci. Here is an example, which relies on the class deci to display the value of an infinint variable:

libdar::statistics report;

    // assuming we have given &report in place of nullptr to libdar::archive operation
    // which is now running in another thread

 std::string value = libdar::deci(report.get_treated()).human();

 std::cout << std::string("Number of file treated so far: ") << value << std::endl;

Some libdar::archive methods (archive restoration, comparison, testing) also return a value of type libdar::statistics which can be used the same way to have a global summary of the operation but only once the operation has completed.

Last point all fields of libdar::statistics are not used for all operations, because they are not always meaningful (like filesystem error while doing a merging operation).
Please refer to the libdar::archive class documentation (Doxygen) or to the <dar/archive.hpp> header file to have the details of class libdar::statistics used fields and their meaning in relation to each libdar::archive operation.


For more detailed information about the API you can build the API documentation from the source code using Doxygen or get it online from dar home page or its mirror site.


15 - Compilation & Linking

Compilation

All the symbols found in the libdar API except the one relative to dar_manager (see below) are defined from <dar/libdar.h>. So you should only need to include this header. If the header file is not located in a standard directory, in order to compile your code, you may need some extra flags to pass to the compiler (like -I/opt/...). The pkg-config tool can help here to avoid system dependent invocation:


> cat my_prog.cpp

#include <dar/libdar.h>

main()
{
   libdar::get_version(...);
   ...
}

> gcc `pkg-config --cflags libdar` -c my_prog.cpp


Linking


Of course, you need to link your program with libdar. This is done by adding -ldar plus other library libdar can rely on like libz, libbzip2, liblzo or libgcrypt, depending on the feature activated at compilation time. Here too, pkg-config can provide a great help to avoid having system dependent invocation:


> gcc pkg-confg --libs libdar` my_prog.o -o my_prog

Libdar's different flavors


Well, all the compilation and linking steps described above assume you have a "full" libdar library. Beside the full (alias infinint) libdar flavor, libdar also comes in 32 and 64 bits versions. In these last ones, in place of internally relying on a special type (which is a C++ class called infinint) to handle arbitrary large integers, libdar32 relies on 32 bits integers and libdar64 relies on 64 bits integers (there are limitations which are described in doc/LIMITATIONS). But all these libdar version (infinint, 32bits, 64bits) have the same interface and must be used the same way, except for compilation and linking.

These different libdar versions can coexist on the same system, they share the same include files. But the LIBDAR_MODE macro must be set to 32 or 64 when compiling or linking with libdar32 or libdar64 respectively. The LIBDAR_MODE macro defines the way the "class infinint" type is implemented in libdar, and thus changes the way the libdar headers files are interpreted by the compiler. pkg-config --cflags will set the correct LIBDAR_MODE, so you should only bother calling it with either libdar, libdar32 or libdar64 depending on your need : "pkg-confg --cflags libdar64" for example.

> cat my_prog.cpp
#include <dar/libdar.h>

main()
{
   libdar::get_version(...);
   ...
}
> gcc -c `pkg-config --cflags libdar32` my_prog.cpp


> gcc `pkg-config --libs libdar32` my_prog.o -o my_prog

and replace 32 by 64 to link with libdar64.





16 - Aborting an Operation

If the POSIX thread support is available, libdar will be built in a thread-safe manner, thus you may have several thread using libdar calls at the same time. You may then wish to interrupt a given thread. But aborting a thread form the outside (like sending it a KILL signal) will most of the time let some memory allocated or even worse can lead to dead-lock situation, when the killed thread was inside a critical section and had not got  the opportunity to release a mutex. For that reason, libdar proposes a set of calls to abort any processing libdar call which is ran by a given thread.

   // next is the thread ID in which we want to have lidbar call canceled
   // here for simplicity we don't describe the way the ID has been obtained
pthread_t thread_id = 161720;
  
   // the most simple call is :
libdar::cancel_thread(thread_id);
   // this will make any libdar call in this thread be canceled immediately

   // but you can use something a bit more interesting:
libdar::cancel_thread(thread_id, false);
   // this second argument is true for immediate cancellation,
   // of false for a delayed cancellation, in which case libdar aborts the operation
   // but produces something usable, for example, if you were backing up something
   // you get a real usable archive which only contains files saved so far, in place
   // of having a broken archive which miss a catalogue at the end. Note that this
   // delayed cancellation needs a bit more time  to complete, depending on the
   // size of the archive under process.

As seen above, cancellation can be very simple. What now succeeds when you ask for a cancellation this way? Well, an exception of type Ethread_cancel is thrown. All along his path, memory is released and mutex are freed. Last, the exception appears to the libdar caller. So, you can catch it to define a specific comportment. And if you don't want to use exceptions a special returned code is used.

try
{
    libdar::archive *my_arch =
             new libdar::archive(...);
    ...
}
catch(libdar::Ethread_cancel & e)
{
    ... do something when thread has been canceled;
}

U_16 ex;
std::string msg;
archive *my_arch =
   libdar::open_archive_noexcept(...,ex,msg);

switch(ex)
{
case ...
  ....
  break;
case LIBDAR_THREAD_CANCEL:
  ... do something when thread has been canceled
  break;
case ...
}

Some helper routines are available to know the cancellation status for a particular thread or to abort a cancellation process if it has not yet been engaged.

 pthread_t tid;
  
   // how to know if the thread tid is under cancellation process ?
if(libdar::cancel_status(tid))
     cout << "thread cancellation is under progress for thread : " << tid << endl;
else
     cout << "no thread cancellation is under progress for thread : " << endl;

   // how to cancel a pending thread cancellation ?
if(libdar::cancel_clear(tid))
    cout << "pending thread cancellation has been reset, thread " << tid << " has not been canceled" << endl;
else
   cout << "too late, could not avoid thread cancellation for thread "<< tid << endl;

Last point, back to the Ethread_cancel exception, this class has two methods you may find useful, when you catch it:

try
{
   ... some libdar calls
}
catch(libdar::Ethread_cancel & e)
{
   if(e.immediate_cancel())
       cout << "cancel_thread() has been called with "true" as second argument" << endl;
   else
      cout << "cancel_thread() has been called with "false" as second argument" << endl;

   U64 flag = e.get_flag();
    ... do something with the flag variable...
}

    // what is this flag stored in this exception ?
    // You must consider that the complete definition of cancel_thread() is the following:
    // void cancel_thread(pthread_t tid, bool immediate = true, U_64 flag = 0);
    // thus, any argument given in third is passed to the thrown Ethread_cancel exception,
    // value which can be retrieved thanks to its get_flag() method. The value given to this
    // flag is not used by libdar itself, it is a facility for user program to have the possibility
    // to include additional information about the thread cancellation.

    // supposing the thread cancellation has been invoked by :
libdar::cancel_thread(thread_id, true, 19);
   // then the flag variable in the catch() statement above would have received
   // the value 19.

A last and important point about multi-threaded environment: An object like any other variable cannot be modified or read (with the use of its methods) without precaution from several threads at the same time. Care must be taken to avoid this situation, and the use of Posix mutex is recommanded in your program if you plan to let an archive object be accessed by more than one thread. See the FAQ for more about this point.




17 - Dar_manager API


For more about dar_manager, please read the man page where are described in detail the available features. Note that for dar_manager there is not a "without exception" flavor, your program must be able to handle exceptions, which by the way are the same as the ones described above.

To get dar_manager features you need to use the class database which is defined in the libdar/database.hpp header file so you first need to include that file. Most of the methods of the database class do use options. For the same reason as previously seen for archive manipulation, these options are passed thanks to a container class. These container classes for options used by the database class are defined in the libdar/database_options.hpp file. Let's see the different method of the class database :

Database object construction

Two constructor are available:

#include <dar/database.hpp>

void my_sample_function(user_interaction & dialog)
{
    database base;   // we have created an empty database (no archive in it) called "base"

    database other  = database(dialog, "/tmp/existing_base.dmd", database_open_options());
                            // we have created a database object called "other" which contains
                            // (in RAM) all information that were contained in the
                            // database file "/tmp/existing_base.dmd"
                            // I will explain below the last argument

    database_open_option opt;
    opt.set_partial(true);
    database other2 = database(dialog, "/tmp/existing_base.dmd", opt);
                           // we have created a database object called "other2" which differs
                           // from "other" in the option we used. While "other" is a fully loaded
                           // database, "other2" is a partial database. This notion is explained
                           // below
}


So far, this is not much complicated. You can build an empty database from nothing, or load a database to memory from a file using the second constructor. As you can see over the filename to give in this later constructor, we need a user_interaction object to be able to inform the user of any problem that could be met, and an object of class database_open_options. This last object contains options to use for this call (options are set to their default unless modified explicitely). Currently, the only available option is the "partial" option which is a boolean argument:

In all the available methods for class database, some require to load the whole database in the memory while some other only require the database header. Loading just the database header is much faster than loading the whole database, of course, and as you guess it requires much less memory. While you can perform any operation with a full loaded database, only a subset of available method will be available with a partially loaded database.  If you try a method that requires a completely loaded database, you will get an exception if the object you use has been loaded with "true" as last argument (called "partial") of the constructor, and of course an empty database (built with the first constructor) is a completely loaded database, so you don't have restriction in using a new database object.

But now let's see the available method for that class:

Database's methods

First we will see methods that work with both partially and completely loaded databases:
  • dump(...) : it is used to write back the database to a file.
  • change_name() : change the basename of the archive which index is given in argument
  • set_path() : change the path to the archive which index is given in argument
  • set_options() : change the default options to always pass to dar when performing restoration
  • set_dar_path() : specify the path to dar (use empty string to rely on the PATH variable)
  • show_contents() : list the archives used to build the database
  • get_options() : list the options that will be passed to dar (as defined with the set_options() method)
  • get_dar_path() : return the path to dar (or empty string if relying on the PATH variable)

Now let's see the database methods that only work with completely loaded databases:
  • add_archive() : add an archive to the database
  • remove_archive() : remove an archive from the database
  • set_permutation() : change archive relative order within the database
  • show_files() : list the files which are present in the given archive
  • show_version() : list the archive where the given file is saved
  • show_most_recent_stats() :  compute statistics about the location of most recent file versions
  • restore() : restore a set of given files given in argument.
Well, you might now say that as description this is a bit light for a tutorial, yes. In fact these call are really very simple to use, you can find a complete description in the reference documentation of the API. This documentation is built if doxygen is available and is put under doc/html after calling make in the source package. It is also available from dar's homepage.


Thanks


I would like to thank Wesley Leggette and Johnathan Burchill for having given their feedback and having done grammar corrections to this document. Out of this document, I would also like to thanks them a second time for their work around dar and libdar (Johnathan is the author of kdar, the KDE front-end for dar).

Regards,
Denis Corbin.