Dar Documentation

LIBDAR

APPLICATION INTERFACE

TUTORIAL

for API version 6.0.x and later


Presentation

The Libdar library provides a complete abstraction layer for handling Dar archives. The general operations provided are:

  • archive creation,
  • file extraction from archive,
  • archive listing,
  • archive testing,
  • archive comparison,
  • catalogue isolation,
  • archive merging,
  • archive reparation
  • dar_manager database manipulations
  • dar_slave steering
  • dar_xform operation
Note that Disk ARchive and libdar have been released under the Gnu General Public License (GPL). All code linked to libdar (statically or dynamically), must also be covered by the GPL. Commercial use is prohibited unless a contract has been agreed with libdar's author.

This tutorial will show you how to use the libdar API.  Since release 2.0.0 the dar command-line executable also relies on this API, looking at it's code may provide a good illustration on the way to use libdar, the file src/dar_suite/dar.cpp is the primary consumer of the libdar API.

The sample codes provided here is solely illustrative and is not guaranteed to compile. More detailed API documentation is contained in the source code and can be compiled to the doc/html directory using Doxygen, which is also provided online and is referred below as the API reference documentation.


Let's Start

Conventions

Language

Dar and libdar are written in C++, and so is the libdar API, for other languages check for the existing bindings, like for example the python bindings.

Header files

Only one include file is required in your program to have access to libdar:

#include <dar/libdar.hpp>

Libdar namespace

All libdar symbols are defined under the libdar namespace. You can either add the using namespace libdar; line at the beginning of your source files:

using namespace libdar;

get_version();

 or, as shown below, you can explicitly use the namespace in front of libdar symbols, we will use this notation in the following:


libdar::get_version();

Exceptions

The library makes use of exception to report unexpected conditions. These contain the reason and context the error occurred in and can be caught by your code to display this information. All exceptions used within libdar inherit from the pure virtual class libdar::Egeneric

Most of the time you will use only one of the following two methods:
  • std::string & get_message() const
  • std::string & dump_str() const
get_message() returns a message string describing the error met
dump_str() returns a text paragraph with additional information about the stack as well as context the error occurred in.

We will only focus on one specific exception type libdar::Ebug that is used within libdar when a-situation-that-should-never-occur is met and is assumed to be a bug in libdar. Using the get_message() method in that situation would not provide all necessary details to understand and fix the bug, so it is advised to always use dump_str() for that specific type of exception.

try
{
    // calls to libdar
   ...
    //
}
catch(libdar::Ebug & e)
{
    std::string msg = e.dump_str();

    // do something with msg like for example:
    std::cerr << msg
}
catch(libdar::Egeneric & e)
{
    std::string msg = e.get_message();

    // do something with msg like for example
    std::cerr << msg
}


 1 - First we must initialize libdar

the libdar initialization is performed by calling the libdar::get_version() function.

This function can be called several time though only once is necessary, but this call has to complete before any other call to libdar.

In a multi-thread context libthreadar initialization is not re-entrant. In other word the first call call to libdar::get_version() must complete before any other call to libdar can take place. Once libdar  has been initialized, you can call libdar::get_version() concurrently from several threads at the same time.


libdar::get_version();

2 - We should prepare the end right now

Libdar used some data-structures (mutex, secured memory, etc.) that need to be released properly before ending the program. It is important to invoke the following function before exiting your program if you invoked get_version() previously.


libdar::close_and_clean()

Note: closes_and_clean() makes the necessary for memory to be released in the proper order. Not calling close_and_clean() at the end of your program may result in uncaught exception message from libdar at the end of the execution. This depends on the compiler, libc and option activated in libdar at compilation time.


All in one, at the highest level, you code should look like the following


try
{
    libdar::get_version();
    // calls to libdar
    // thing we will see in next
       ...
       ...
   
}
catch(libdar::Ebug & e)
{
    std::string msg = e.dump_str();

    // do something with msg like for example:
    std::cerr << msg
}
catch(libdar::Egeneric & e)
{
    std::string msg = e.get_message();

    // do something with msg like for example
    std::cerr << msg
}

libdar::close_and_clean();


3 - Intercepting signals

libdar by itself does not make use of any signal (see signal(2) and kill(2)). However, gpgme library with which libdar may be linked with in order to support asymmetrical strong encryption (i.e. encryption using public/private keys) may trigger the PIPE signal. Your application shall thus either ignore it (signal(SIGPIPE, SIG_IGN)) or provide an adhoc handle. By default the PIPE signal leads the receiving process to terminate.


4 - Libdar classes

The main components of libdar are four classes:
  • class libdar::archive to play with dar archives
  • class libdar::database to play with dar_manager databases
  • class libdar::libdar_slave to take the role of dar_slave
  • class libdar::libdar_xform to re-slice existing archives like dar_xform does
In the following we will first see class libdar::archive which will take most of our effort as other classes which we will see at the end are very simple to use.

5 - Multithreaded environment

Except some specific cases that will be mentioned, a given libdar object can only be manipulated by a single thread. You can however perform several operations concurrently from different thread, each having its own set of objects. Though, if one thread is creating an archive by mean of an first object and at the same time another thread by mean of a second object is trying to read the same archive under construction, things might not work as expected. But this is obvious considerations we will not dig any further assuming you know what you are doing.

6 - Let's create a simple archive

Creating a libdar::archive object depending on the constructor used, leads to either:
  • the creation of a brand new archive on filesystem, thus performing a backup (full, incremental, differential, decremental, ...)
  • the opening an archive existing on filesystem, for further operation (listing,  file restoration,  archive testing, archive difference, ...)
  • the merging two existing archives into a new one
  • the reparation of an archive which catalogue is missing or damaged. The catalogue (which means catalog in French) is the table of content of an archive.

6.1 - Basic archive creation

For archive creation the constructor format is the following one:

        archive::archive(const std::shared_ptr<user_interaction> & dialog,
                         const path & fs_root,
                         const path & sauv_path,
                         const std::string & filename,
                         const std::string & extension,
                         const archive_options_create & options,
                         statistics * progressive_report);

For now we will left beside several parameters seen in detail later:
  • dialog can be set to std::nullptr for now, this means that all interaction with the user will be done by mean of standard input, output and error.
  • fs_root is the directory to take as root of the backup. The libdar::path class can be setup from a std::string
  • sauv_path is the path where to write the archive to, here also a std::string will do the job
  • filename is the slice name of the archive to create
  • extension is the archive extension to use. There is no reason to not use the string "dar" here
  • options is a class that carries all optional parameters, it contains a constructor without argument so all options are set to their default that way
  • statistics can receive the address of an existing object that another thread can read while a first one is doing a backup operation, we will see this feature later on, but for now let's set this to a null pointer (i.e.: std::nullptr)
Once the object has been created (the constructor has returned), the archive operation has completed and a new file has been completely written on disk.

libdar::archive my_first_backup(nullptr,
                                "/home/me",
                                "/tmp",
                                "first_backup",
                                "dar",
                                archive_options_create(),
                                nullptr);

Thus, the previous command will create a single sliced archive "first_backup.1.dar" located under /tmp. It will contain the content of the directory /home/me and its sub-directories, without compression and without ciphering. You have guessed compression, slicing, ciphering can be set by playing with passing an adhoc archive_option_create object to this archive constructor, we will see that later.

Once the object has been created there is only little thing we can do with it, like archive listing or archive isolation. But archive extraction, testing or diffing needs  creating a object with a "read" constructor first.

We could also have allocated the archive on the heap, in that case we would have just added the delete operation after the construction has ended:

libdar::archive* my_first_backup = new libdar::archive(nullptr,
                                                                        "/home/me",
                                                                        "/tmp",
                                                                        "first_backup",
                                                                        "dar", 
                                                                         archive_options_create(),
                                                                          nullptr);
  // we assume std::bad_alloc would be thrown if an allocation problem had occurred
  // same thing if libdar throws an exception at constructor time, the object would
  // not be created and would not have to be deleted.
  // So now we can delete the created object:

delete my_first_backup;

6.2 - Progressive report

During the operation we get nothing shown unless an error occurs. To have more visibility on the process we will use an libdar::statistics object passed as last argument of this constructor. The useful method of class libdar::statistics are:
  • std::string get_treated_str()
  • std::string get_hard_links_str()
  • std::string get_skipped_str()
  • std::string get_inode_only_str()
  • std::string get_ignored_str()
  • std::string get_tooold_str()
  • std::string get_errored_str()
  • std::string get_deleted_str()
  • std::string get_ea_treated_str()
  • std::string get_byte_amount_str()
  • std::string get_fsa_treated_str()
If you have a doubt about the meaning and use of a particular counter in a particular operation, please refer to API reference documentation of class libdar::statistics, the private fields corresponding to these counter are explicitly defined there.

libdar::statistics stats;

libdar::archive my_first_backup(nullptr,
                                "/home/me",
                                "/tmp",
                                "first_backup",
                                "dar",
                                archive_options_create(),
                                & stats);

std::cout << stats.get_treated_str() << " file(s) saved" << std::endl;
std::cout << stats.get_errored_str() << " file(s) failed to backup" << std::endl;
std::cout << stats.get_ea_treated_str() << " Extended Attributes saved" << std::endl;


6.3 - Archive creation options

in the previous example, we have created an object of class libdar::archive_options_create and passed it on-fly to the archive constructor without modifying it. Thus we used the default options for this operations. But a lot of options are available, each one can be modified by a specific method, you will quite never use all of them. Follow is a subset of the available options. We won't details them all, but you can refer the doxygen documentation of class libdar::archive_options_create for more information.
  • void set_reference(std::shared_ptr<archive> ref_arch)
  • void set_selection(const mask & selection)
  • void set_subtree(const mask & subtree)
  • void set_allow_over(bool allow_over)
  • void set_warn_over(bool warn_over)
  • void set_info_details(bool info_details)
  • void set_display_treated(bool display_treated, bool only_dir)
  • void set_display_skipped(bool display_skipped)
  • void set_display_finished(bool display_finished)
  • void set_pause(const infinint & pause)
  • void set_empty_dir(bool empty_dir)
  • void set_compression(compression compr_algo)
  • void set_compression_level(U_I compression_level)
  • void set_slicing(const infinint & file_size, const infinint & first_file_size)
  • void set_ea_mask(const mask & ea_mask)
  • void set_execute(const std::string & execute)
  • void set_crypto_algo(crypto_algo crypto)
  • void set_crypto_pass(const secu_string & pass)
  • void set_compr_mask(const mask & compr_mask);
  • void set_min_compr_size(const infinint & min_compr_size)
  • void set_nodump(bool nodump)
  • void set_exclude_by_ea(const std::string & ea_name)
  • void set_what_to_check(comparison_fields what_to_check)
  • void set_hourshift(const infinint & hourshift)
  • void set_empty(bool empty)
  • void set_alter_atime(bool alter_atime)
  • void set_furtive_read_mode(bool furtive_read)
  • void set_same_fs(bool same_fs)
  • void set_snapshot(bool snapshot)
  • void set_cache_directory_tagging(bool cache_directory_tagging)
  • void set_fixed_date(const infinint & fixed_date)
  • void set_slice_permission(const std::string & slice_permission)
  • void set_slice_user_ownership(const std::string & slice_user_ownership)
  • void set_slice_group_ownership(const std::string & slice_group_ownership)
  • void set_retry_on_change(const infinint & count_max_per_file, const infinint & global_max_byte_overhead)
  • void set_security_check(bool check)
  • void set_user_comment(const std::string & comment)
  • void set_hash_algo(hash_algo hash)
  • void set_slice_min_digits(infinint val)
  • void set_backup_hook(const std::string & execute, const mask & which_files);
  • void set_delta_diff(bool val)
  • void set_delta_signature(bool val)
  • void set_delta_mask(const mask & delta_mask)
First you may have find some strange types in arguments, we will briefly explain how to set them:
  • std::shared_ptr<archive> C++11 shared smart-pointer to an existing archive object. We will see how to use it next when performing differential backup
  • infinint can be set from a classical unsigned int, unsigned long or other unsigned integer type
  • mask is a bit more complex, we will see how to use it in a specific paragraph of this tutorial
  • compression is an enumeration with values like
    • libdar::compression::gzip
    • libdar::compression::bzip2
    • libdar::compression::xz
    • libdar::compression:lzo
  • U_I is to be considered an unsigned int
  • crypto_algo is also an enumeration with values like
    • libdar::crypto_algo::scrambling
    • libdar::crypto_algo::blowfish
    • libdar::crypto_algo::aes256
    • libdar::crypto_algo::twofish256
    • libdar::crypto_algo::serpent256
    • libdar::crypto_algo::camellia256
  • secu_string is a class securely storing password and sensible cryptographic information. It can be setup from a char* or from a filedescriptor. its main constructor is:
    • secu_string(const char* ptr, U_I size)
  • comparison_fields is an enumeration with values like
    • libdar::comparison_fields::all
    • libdar::comparison_fields::ignore_owner
    • libdar::comparison_fields::mtime
    • libdar::comparison_fields::inode_type

libdar::archive_options_create opt;

opt.set_allow_over(false);       // forbids slice overwriting
opt.set_display_finished(true);  // show a summary after each completed directory
opt.set_slicing(1024000, 2000);  // slices of 1000 kiB initial slice of 2000 bytes
opt.set_pause(2);                // pause every two slices
opt.set_execute("echo slice %N completed"); // command executed after each slice
opt.set_crypto_algo(libdar::crypto_algo::aes256);

 // providing an empty secu_string leads dar interactively ask the passphrase in a secure manner
opt.set_crypto_pass(secu_string());
 // this previous call is useless as en empty secu_string is the default
 // though one could have setup a secu_string from a std::string this way:
std::string my_pass("hello world!");
libdar::secu_string my_secupass(my_pass.c_str(), my_pass.size());
opt.set_crypto_pass(my_secupass);

opt.set_compression(libdar::compression::xz);
opt.set_compression_level(9);   // this is the default
opt.set_min_compr_size(10240);  // not trying compressing file smaller than 10 kiB

libdar::archive my_first_backup(nullptr,
                                "/home/me",
                                "/tmp",
                                "first_backup",
                                "dar",
                                opt,
                                nullptr);

Of course, you can use both libdar::statistics and libdar::archive_options_create at the same time.

7 - Creating a differential or incremental backup

Maybe you have guessed? Compared to the previous operation (full backup) doing an differential or incremental backup will only ask to open in read-mode an existing archive and pass this object as argument of class archive_options_create::set_reference() seen just above.

The read-only constructor for class archive is the following:

    archive(const std::shared_ptr<user_interaction> & dialog,
            const path & chem,
            const std::string & basename,
            const std::string & extension,
            const archive_options_read & options);

same as before:
  • dialog can be set to a null pointer, we will see later on how to play with user_interaction class
  • chem is the path leading to the archive to read, it can be provided as a std::string
  • basename is the archive basename to read
  • extension should be "dar" unless you want to confuse people
  • options can be set to an empty object for default options, we will see this class in more details with archive listing

  // first we open the previously created archive in read mode:

std::shared_ptr<libdar::archive> ref_archive(new libdar::archive(nullptr,
                                                  "/home/me",
                                                  "first_backup",
                                                  "dar",
                                                  archive_create_options_read()));

  // here we dynamically allocated the object to be able to setup
  // a shared_ptr in order to passed it to
  // the archive_options_create::set_reference() method:

libdar::archive_options_create opt;
opt.set_reference(ref_archive);

libdar::archive my_second_backup(nullptr,
                                "/home/me",
                                "/tmp",
                                "diff_backup",
                                "dar",
                                opt,
                                nullptr);

creating a incremental backup is exactly the same, the difference is the nature of the archive of reference. We used to describe a differential backup one that has a full backup as reference, while an incremental backup has another incremental or differential backup as reference (not a full backup).

8 - Archive listing

Archive listing operation consist of the creation of an archive object in read-mode as we just did above and invoking a method on that newly object to see all or a sub-directory content of the archive. Before looking at the listing method let's zoom on the class libdar::archive_create_options_read we just skip over previously.

8.1 - Archive reading options

 The same as the class archive_options_create detailed above, the class archive_options_read has a constructor without argument that sets the different options to their default value. You can change them one by one by mean of specific methods. The most usual ones are:
  • void set_execute(const std::string & execute)
  • void set_info_details(bool info_details)
  • void set_sequential_read(bool val)
  • void set_slice_min_digits(infinint val)
set_execute() has runs a command before reading a new slice of the archive. See API reference documentation for details. You will meet that class in order to test an archive, compare an archive with filesystem, isolate an archive and repair an archive.

8.2 - Listing methods

There is several way to read an given archive contents:
  • void op_listing(archive_listing_callback callback,
                            void *context,
                            const archive_options_listing & options) const;
  • bool get_children_of(archive_listing_callback callback,
                                 void *context,
                                 const std::string & dir,
                                 bool fetch_ea = false);
  • const std::vector<list_entry> get_children_in_table(const std::string & dir, bool fetch_ea = false) const
archive::op_listing() makes use of a callback function that will be called in turn for each entry of the archive even special entries that flag the end of a directory and the next entry will be located in the parent directory.
archive::get_children_of() use the same callback but only for the different entries of a given directory, that has to exist in the archive of course. It returns false when the end of the directory has been reached.
archive::get_children_in_table() is like the previous listing a given directory but returns a vector of objects libdar::list_entry that provide detailed information about each entry, no callback is used here.

For the two first methods you have to define a callback function of the following form

void (*)(const std::string & the_path,
         const list_entry & entry,
         void *context);

This callback will receive as argument the full path of the object, a libdar::list_entry object providing much details on it and the "context" value passed as argument of archive::op_listing() or archive::get_children_of()

Last point to see before going forward with an example is this libdar::list_entry class, we will use here only a few of the rich set of fields/methods this class provides in the following examples:

  // we first create a read-mode archive object that will be used in the three following examples
 // we will also illustrate the use of libdar::archive_options_read

libdar::archive_options_read opt;

opt.set_info_details(true);
opt.set_execute("echo 'about to read slice %p/%b.%N.%e with context %c'");

libdar::archive my_backup(nullptr,   // this is user_interaction we will see further
                          "/home/me",
                          "diff_backup",
                          "dar",
                           opt);


  // we will also need a backup function for the two first methods
  // let's define it:

void my_listing_callback(const std::string & the_path,
                         const libdar::list_entry & entry,
                         void *context)
{
    std::cout << the_path;
    if(entry.is_dir())
       std::cout << " is a directory";
    std::cout << " with permission " << entry.get_perm();
    std::cout << " located in slices " << entry.get_slices().display();
    std::cout << std::endl;
     // yep, we do not need context, this
     // is available if you need it though

    if(entry.is_eod())
    {
       // only op_listing() provides such type of object
       // which occurs when we reached the End Of Directory
       // next entry will be located in the parent directory.
       //
       // Note for op_listing: when reading a directory we recurs in it,
       // meaning that the next entry this callback will be
       // invoked for will be located in that directory
       //
       // for get_children_of() no recursion or eod object is
       // performed about directory. The next entry following
       // a directory is still located in the same parent directory
       // which when fully read stops the get_children_of() routine
       // at the difference of op_listing() which parse the whole
       // directory tree.
       //
       // For example, reading a empty directory will provide
       // that directory info, then an eod object a the next
       // callback invocation.
    }
}


8.3 - archive listing using archive::op_listing()

  // first possibility: we can pass nullptr as callback function to archive::op_listing, all will be displayed in stdout

my_backup.op_listing(nullptr, // no callback function
                     nullptr, // we don't care of context here
                     archive_options_listing()) // and use default listing options

  // second possibility: we use the callback defined above

my_backup.op_listing(my_listing_callback,
                     nullptr, // we still don't care of context here
                     archive_options_listing()) // and still the default listing options

  // in complement of both previous variant we can of course set non default listing options

libdar::archive_options_listing opt;

opt.set_filter_unsaved(true); // skip entry that have not been saved since the archive of reference
opt.set_slice_location(true); // necessary if we want to have slicing information available in the callback function
opt.set_fetch_ea(false);  // this is the default. Set it to true if you want to use list_entry::get_ea_reset_read()/get_ea_next_read()

my_backup.op_listing(my_listing_callback,
                     nullptr, // we still don't care of context here
                     opt) // and still the default listing options


8.4 - archive listing using archive::get_children_of()

  // With this method we only list one directory

my_backup.get_children_of(my_listing_callback,
                          nullptr,  // we still don't care of context here
                          "",       // we read the root directory of the archive
                          true);    // and ask for EA retrieval, but as we do not
                                    // use list_entry::get_ea_read_next() in the
                                    // callback this is just wasting CPU and memory

or course if you have a sub-directory /home/me/.gnupg/private-keys-v1.d in your home directory and you want to check how it is saved in the archive, as we defined the root of the backup as /home/me and as you always have to pass a relative path (no leading /) you could do that by calling the following:

my_backup.get_children_of(my_listing_callback,
                          nullptr,
                          ".gnupg/private-keys-v1.d");                    



8.5 - archive listing using archive::get_children_in_table()

// still listing a single directory but this time without callback function:

my_backup.init_catalogue(); // necessary to fill read the whole catalogue in memory
                            // in particular if archive has been opened in sequential read mode

std::vector<libdar::list_entry> result = my_backup.get_children_in_table(".gnupg/private-keys-v1.d");

// now reading the std::vector

std::vector<libdar::list_entry>::iterator it = result.begin();
while(it != result.end())
{
   if(it->is_dir())
       std::cout << " is a directory";
    std::cout << " with permission " << it->get_perm();
    std::cout << " located in slices " << it->get_slices().display();
    std::cout << std::endl;
}


9 - Testing an archive

As seen for listing operation we assume a archive object has been create in read mode. Testing the coherence of the relative archive files on disk is done by calling the libdar::op_test method:

        statistics op_test(const archive_options_test & options,
                           statistics * progressive_report);

You may recognize the libdar::statistics type we saw for archive creation. It is present as argument and the provided libdar::statistics object can be read during the whole testing operation by another thread. But if you just want the to know the result, you'd better just use the returned value as it makes the operation quicker due to the absence of multithread management.


   // for the exercise, we will change some default options:

archive_options_test opt;
opt.set_info_details(true); // to have a verbose output

libdar::statistics stats;
stats = my_backup.op_test(nullptr,   // still the user_interaction we will see further
                          opt;       // the non default options set above
                          nullptr);  // we will just use the returned value

std::cout << stats.get_treated_str() << " file(s) tested" << std::endl;
std::cout << stats.get_errored_str() << " file(s) with errors" << std::endl;
std::cout << stats.get_ea_treated_str() << " Extended Attributes tested" << std::endl;

10 - Comparing an archive

As simple as previously, but using the archive::op_diff method:

        statistics op_diff(const path & fs_root,
                           const archive_options_diff & options,
                           statistics * progressive_report);
      

Over the type of the option field, you see the fs_root argument which define which directory of the filesystem to compare the archive to


   // for the exercise, we will change some default options:

archive_options_diff opt;
opt.set_info_details(true); // to have a verbose output
opt.set_what_to_check(libdar::comparison_fields::ignore_owner);
      // this option above will consider equal two files which
      // only change due to user or group ownership difference
      // by default any difference will be considered a difference
(void) my_backup.op_diff("/home/me",
                          opt;       // the non default options set above
                          nullptr);  // not using it for this example


11 - Isolating an archive

As simple as previously, but using the archive::op_isolate method:

        void op_isolate(const path &sauv_path,
                        const std::string & filename,
                        const std::string & extension,
                        const archive_options_isolate & options);
      
You will find similitude with the archive creation though here this is not a constructor

sauv_path is the directory where to create the isolated version of the current archive
filename is the archive basename to create
extension should still be "dar" here too
options are options for isolation like slicing, compression, encryption similar to the archive_options_create class we saw at the beginning of this tutorial

   // for the exercise, we will change some default options:

archive_options_isolate opt;
opt.set_warn_over(false); 
   // by default overwriting is allowed by a warning is issued first
   // here overwriting will take place without warning

opt.set_compression(libdar::compression::gzip);
opt.set_compression_level(9);   // this is the default
opt.set_min_compr_size(10240);  // not trying compressing file smaller than 10 kiB

my_backup.op_isolate("/tmp",
                     "CAT_diff_backup",
                     "dar",
                      opt); // the non default options set above
 
   // have you noted? There is no libdar statistics field returned nor as argument.

12 - Restoring files from an archive

Quite as simple as previously, here we use the archive::op_extract method:

        statistics op_extract(const path &fs_root,
                              const archive_options_extract & options,
                              statistics *progressive_report);


      
fs_root is the directory under which to restore the files and directory
options defines how and what to restore
progressive_report has already been seen several time previously

   // as we still do not have seen masks, we will restore all files contained in the backup
   // such mask would be provided to the
   // archive_options_extract::set_selection() and/or
   // to the archive_options_extract::set_subtree() methods
   // to precisely define what files to restore

archive_options_extract opt;

opt.set_dirty_behavior(false, false); // dirty files are not restored

(void) my_backup.op_extract("/home/me/my_home_copy",
                            opt,
                            nullptr); // we have seen previously how to use statistics
 


13 - Merging archives

Here we will need two archive objects open in read-mode and we will invoke a specific archive constructor passing these two objects as argument, once the constructor will have completed the merging operation will be done:


        archive(const std::shared_ptr<user_interaction> & dialog,
                const path & sauv_path,
                std::shared_ptr<archive> ref_arch1,
                const std::string & filename,
                const std::string & extension,
                const archive_options_merge & options,
                statistics * progressive_report);
      
dialog is will still be set to null pointer for now
sauv_path is the directory where to create the resulting merging archive
ref_arch1 is the first (and mandatory) archive, the second is optional and may be given to the options argument
filename is the resulting archive basename
extension as always should be set to "dar"
options is a set of optional parameters
progressive_report is as seen above the ability to have another thread showing progression info during the operation

  // assuming you have two backups:
  // the first is /tmp/home_backup.*.dar
  // the second is /var/tmp/system_backup.*.dar
  // we will create /tmp/merged.*.dar as result of the merging
  // of these two backups

  // 1 - first things first: opening the first backup

 libdar::archive_options_read opt;

opt.set_info_details(true);
opt.set_execute("echo 'about to read slice %p/%b.%N.%e with context %c'");

std::shared_ptr<libdar::archive> home_backup(new libdar::archive(nullptr,   // this is user_interaction we will see further
                                                      "/tmp",
                                                      "home_backup",
                                                      "dar",
                                                       opt));

  // 2 - opening the second backup

std::shared_ptr<libdar::archive> system_backup(new libdar::archive(nullptr,
                                                       "/var/tmp",
                                                       "system_backup",
                                                       "dar",
                                                       opt);

  // 3 - setting up the options for merging

libdar::archive_options_merge opt_merge;

opt_merge.set_auxiliary_ref(system_backup);
opt_merge.set_slicing(1048576, 0); // all slice would have 1 MiB at most
opt_merge.set_compression(libdar::compression::bzip2);
opt_merge.set_keep_compressed(true);
opt_merge.set_user_comment("archive resulting of the merging of home_backup and system_backup");
opt_merge.set_hash_algo(libdar::hash_algo::sha512); // will generate on-fly hash file for each slice

  // 4 - now performing the merging operation

libdar::archive merged(nullptr,  // still the user_interaction we will see further
                       "/tmp",
                       home_backup,
                       "merged",
                       "dar",
                       opt_merge,
                       nullptr);  // progressive_report we don't use here


14 - Decremental backup

Decremental backup is an operation that from two full backups an old and a recent one creates a backward differential backup corresponding to the old full backup based on the new full backup. In other words, instead of keeping two full backups, you can keep the latest and replace the oldest by its decremental counterpart. This will save you space while letting you restore as if you had the old full backup by restoring first the recent backup then the decremental backup.

Creating a decremental backup is exactly the same as creating a merging backup, you need just to set the archive_options_merge::set_decremental_mode() before proceeding to the merging. To avoid duplication we will just illustrate the last step of the previous operation modified for decremental backup:

  // [...]

libdar::archive_options_merge opt_merge;

  // [...]
opt_merge.set_decremental_mode(true);

  // 4 - now performing the merging operation (here decremental backup)

libdar::archive merged(nullptr,  // still the user_interaction we will see further
                       "/tmp",
                       home_backup,
                       "merged",
                       "dar",
                       opt_merge,
                       nullptr);  // progressive_report we don't use here



15 - Archive repairing

If an archive has been truncated due to lack of disk space and if sequential marks (aka tape marks) had not been disable, it is possible to rebuild sane archive beside this truncated one.

We just need to invoke a specific libdar::archive constructor which form follows:

        archive(const std::shared_ptr<user_interaction> & dialog,
                const path & chem_src,
                const std::string & basename_src,
                const std::string & extension_src,
                const archive_options_read & options_read,
                const path & chem_dst,
                const std::string & basename_dst,
                const std::string & extension_dst,
                const archive_options_repair & options_repair);

You should now be familiarized with the different types and variable uses. As you can note, this constructor takes in charge the work to read the damaged archive, so you won't have to do it first. As always, this constructor will end only once the operation will have completed.

  // assuming the archive /tmp/home_backup.*.dar is damaged
  // and you want to have repaired archive as /tmp/home_backup_repaired.*.dar

libdar::archive repaired(nullptr,   // still the user_interaction we have not yet seen
                         "/tmp"
                         "home_backup",
                         "dar",
                         archive_options_read(),
                         "/tmp",
                         "home_backup_repaired",
                         "dar",
                         archive_options_repair());

  // we have not done fancy things with the two option classes, but we did above
  // enough time for you get all the necessary information from the API reference
  // documentation


16 - Looking at some details

we have covered the different operations the class libdar::archive can be used for, still remains some concepts to details:
  • user_interaction
  • masks
  • how to cleanly interrupt an running libdar routine
  • how to known which compile-time feature has been activated
Then we will see the three other more simple classes :
  • class database
  • class libdar_slave
  • class libdar_xform
For now, maybe you remember that we had to initialize libdar before use, by calling libdar::get_version()? This routine also exists with arguments that will provide as its name suggests the libdar version:

void get_version(U_I & major, U_I & medium, U_I & minor, bool init_libgcrypt = true);

It is advised to use this form to fetch the libdar version major, medium and minor numbers for the following reasons:

you should check that the library you've dynamically linked with is compatible with the features you will be using. The major number must be the same, for no compatibility is assured between two libdar versions of different major numbers. While run-time compatibility is assured between medium numbers, the medium number must be greater or equal to the one used at compilation time to be sure that all the features you want are available in the libdar library you dynamically linked with. Changes between minor versions correspond to bug fixes and is not to imply any API change, thus no constraints is present there (just note the presence of more bugs in lower numbers).

If you use libgcrypt beside libdar in your application you should initialize libgcrypt and not let it be done by libdar the latest argument of this form should be set to false in that case. Note that libgcrypt documentation indicates that libgcrypt must be initialized directly from the application not from an intermediate library.

Follows an example of test that can be performed while initializing libdar:

U_I major, medium, minor;

libdar::get_version(major, medium, minor);

 if(maj != libdar::LIBDAR_COMPILE_TIME_MAJOR ||
   med < libdar::LIBDAR_COMPILE_TIME_MEDIUM)
{
    std::cout << "libdar version we link with is too old for this code" << std::endl;
    // throw an exception or anything else appropriate to that condition
}

17 - checking compile-time features activation

once we have called one of the get_version* function it is possible to access the list of features activated at compilation time thanks to a set of function located in the compile_time nested namespace inside libdar:



void my_sample_function()
{
      
bool ea = libdar::compile_time::ea();
bool largefile = libdar::compile_time::largefile();
bool nodump = libdar::compile_time::nodump();
bool special_alloc = libdar::compile_time::special_alloc();
U_I bits = libdar::compile_time::bits();
// bits is equal to zero for infinint,
// else it is equal to 32 or 64 depending on
// the compilation mode used.

bool thread = libdar::compile_time::thread_safe();
bool libz = libdar::compile_time::libz();
bool libbz2 = libdar::compile_time::libbz2();
bool liblzo = libdar::compile_time::liblzo();
bool libxz = libdar::compile_time::libxz();
bool libcrypto = libdar::compile_time::libgcrypt();
bool furtive_read = libdar::compile_time::furtive_read();

/ // for details see the compile_time namespace in the API reference documentation

}

18 - User Interaction

we have seen std::shared_pointer on class libdar::user_interaction previously but did not used this feature.

18.1 - Defining your own user_interaction class

class libdar::user_interaction defines the way libdar interact with the user during an operation, like an archive creation, restoration, testing and so on. Only four types of interaction are used by libdar:

        void message(const std::string & message);
        void pause(const std::string & message);
        std::string get_string(const std::string & message, bool echo);
        secu_string get_secu_string(const std::string & message, bool echo);

By default an inherited class of libdar::user_interaction called libdar::shell_interaction is used and implements these four type of exchange by mean of text terminal:
  • message() sends the std::string provided by libdar to stdout
  • pause() does the same and ask for the user to press either return or escape to answer yes or no
  • get_string() reads a string from stdin
  • get_secu_string() reads a string into a secu_string object from stdin too
For a GUI you will probably not want stdin and stdout to be used. For that you have the possibility to implement your own inherited class from user_interaction. It should look like the following:

class my_user_interaction: public libdar::user_interaction
{
protected:
      // display of informational message
   virtual void inherited_message(const std::string & message) override;

      // display of a question and returns the answer from user as true/false
   virtual bool inherited_pause(const std::string & message) override;

      // display the message and returns a string from the user, with or without display what the user typed (echo)
   virtual std::string inherited_get_string(const std::string & message, bool echo) override;

      // same as the previous be the user provided string is returned as secu_string
   virtual secu_string inherited_get_secu_string(const std::string & message, bool echo) override;
};

18.2 - Relying on the pre-defined user_interaction_callback class

As an alternative to defining your own inherited class from libdar::user_interaction, libdar provides a class called user_interaction_callback which is an implementation of the user interaction, based on callback functions.

You will need to implement four callback functions:

        using message_callback = void (*)(const std::string &x, void *context);
        using pause_callback = bool (*)(const std::string &x, void *context);
        using get_string_callback = std::string (*)(const std::string &x, bool echo, void *context);
        using get_secu_string_callback = secu_string (*)(const std::string &x, bool echo, void *context);

Then you can create an libdar::user_interaction_callback object using this constructor:

        user_interaction_callback(message_callback x_message_callback,
                                  pause_callback x_answer_callback,
                                  get_string_callback x_string_callback,
                                  get_secu_string_callback x_secu_string_callback,
                                  void *context_value);

Here follows an example of use:

void my_message_cb(const std::string & x, void *context)
{
    cout << x << endl;
}

bool void my_pause_cb(const std::string & x, void *context)
{
   char a;

   cout << x << endl;
   cin >> a;
   return a == 'y';
}

std::string my_string_cb(const std::string & x, bool echo, void *context)
{
  // to be defined
}

libdar::secu_string my_secu_string_cb(const std::string & x, bool echo, void *context)
{
  // to be defined
}

   // eventually using a context_value that will be passed to the callback of the object
void *context_value = (void *)(& some_datastructure);

std::shared_ptr<libdar::user_interaction> my_user_interaction(new libdar::user_interaction_callback(my_message_cb,
                                                                                                    my_pause_cb,
                                                                                                    my_string_cb,
                                                                                                    my_secu_string_cb,
                                                                                                    context_value));

You will also find predefined classes like libdar::user_interaction_blind which always says no in name of the user displays nothing and provide empty strings, as well as libdar::shell_interaction_emulator which given a user_interaction object send to it formatted information as if it was a shell_interaction object, leading one to emulate libdar default behavior under any time of "terminal".

IMPORTANT: all libdar::user_interaction inherited classes provided by libdar are not designed to be manipulated by more than one thread at a time. The use of std::shared_ptr is only here to let the caller not have to manage such object and let libdar release it when no more needed or to let the caller to reuse the same user_interaction object for a subsequent call to libdar which would not be possible if a std::unique_ptr was used.

Now if you design your own user_interaction inherited class and provide them mecanism (mutex, ...) that allow them to be used simultaneously by several thread there is no issue to give pass such one object as argument to different libdar object used by different threads.


19 - Masks

Mask are used to define which string will be considered and which will not. Libdar implements masks as several classes that all inherit from a virtual class that defines the way masks are used. This root class is the class mask and provides the mask::is_covered(const std::string & expression) method which libdar uses to determine which string are considered for an operation which are not.

Strings applied to masks may correspond to filename only, to full path or maybe to other things. That's in the context where the mask is used that the string meaning take place.

 There is several different basic masks classes you can use to build fairly complex masks, while it is possible you should not need to define you own mask classes, if the need arises, please contact libdar developer if you thing an additional class should take place beside the following ones:

class libdar::bool_mask
 boolean mask, either always true or false, it matches either all files or no files at all
class libdar::simple_mask
 matches as done by the shell on the command lines (see "man 7 glob")
class libdar::regular_mask
 matches regular expressions (see "man 7 regex")
class libdar::not_mask
 negation of another mask
class libdar::et_mask
 makes an *AND* operator between two or more masks
class libdar::ou_mask
 makes the *OR* operator between  two or more masks
class lbdar::simple_path_mask

matches if it is subdirectory of mask or is a directory that contains the specified path itself
class libdar::same_path_mask
 matches if the string is exactly the given mask (no wild card expression)
class libdar::exclude_dir_mask
 matches if string is the given string or a sub directory of it
class libdar::mask_list
 matches a list of files defined in a given file

Let's play with some masks :


      // all files will be elected by this mask
  libdar::bool_mask m1(true);   

      // all string that match the glob expression "A*~" will match.
      // the second argument of the constructor tell whether the match is case sensitive so here
      // any file beginning by 'A' or by 'a' and ending by '~' will be selected by this mask:
  libdar::simple_mask m2(std::string("A*~"), false);

      // m3 is the negation if m2. This mask will thus match
      // any string that does not begin by 'A' or 'a' or finishing by '~'
  libdar::not_mask m3(m2);

      // this mask matches any string that is a subdirectory of "/home/joe"
      // and any directory that contains /home/joe, meaning
      // "/", "/home", "/jome/joe" and any subdirectory are matched.
      // here, the second argument is also case sensitivity (so
      //  "/HoMe" will not be selected by this mask as we set it to "true".
  libdar::simple_path_mask m4 = simple_path_mask("/home/joe", true);

      // now let's do some more complex things:
      // m5 will now match only strings that are selected by both m2 AND m4
  libdar::et_mask m5;
  m5.add_mask(m2);
  m5.add_mask(m4);
     
      // we can make more interesting things like this, where m5 will select files
      // that match m2 AND m4 AND m3. But m3 = not m2 so now m5 will never
      // match any file...
  m5.add_mask(m3);

      // but we could do the same with an "ou_mask" and would get a silly
      // counterpart of m1 (a mask that matches any files)
  libdar::ou_mask m6;
  m6.add_mask(m2);
  m6.add_mask(m4);
  m6.add_mask(m3);

      // lastly, the NOT, AND and OR operation can be used recursively.
      // Frankly, it's possible to have masks referring each other!
  libdar::not_mask m7(m6);
  m6.add_mask(m7);

The idea here is not to create object manually, but to link their creation to the action and choices the user makes from the user interface (Graphical User Interface of your application, for example)

Now that you've seen the power of these masks, you should know that in libdar masks are used at several places:
  • A first place is to select files against their names (without path information) this the argument of the set_selection() method of libdar::archive_options_* classes. The mask here does not apply to directories.
  • A second place is to select files against their path+name and it applies here to all type of files including directories, this is the argument of the set_subtree() method of libdar::archive_options_* classes. So with it, you can prune directories, or in any other way restrict the operation to a particular subdirectory, as well as to a particular plain file for example. Important note about this second mask: what your own mask will be compared to by libdar is the absolute path of the file under consideration. If you want to exclude /usr/local/bin from the operation whatever is the fs_root value (which correspond the -R option of dar) using here a libdar::simple_mask("/usr/local/bin") as argument of libdar::archive_options_*::get_subtree() will do the trick.
An exception is the archive testing operation, which has no fs_root argument (because the operation is not relative to an existing filesystem), however the subtree argument exist to receive a mask for comparing the path of file to include or exclude from the testing operation. In this case the situation is as if the fs_root was set to the value "<ROOT>". For example, masks will be compared to "<ROOT>/some/file" when performing an archive test operation.

instead of using explicit string "<ROOT>" you can use libdar::PSEUDO_ROOT const std::string variable
  • A third place concerns Extended Attributes (EA), this is the argument of the set_ea_mask() method of archive_options classes. It is applied to the full EA name in the form <domain>.<name> where <domain> is any string value like but not limited to the usual "user" or "system" domains.
  • A fourth place concerns the file to compress or to avoid compressing. This is the argument of the set_compr_mask() method of libdar::archive_options_* classes. it is works the same as set_selection() seen above, based only to filename without any path consideration.
  • A fifth place concerns files that have need be prepared for backup, this is the argument of the set_backup_hook() method of libdar::archive_option_create class. I has to be used the same as set_subtree(). For more about this feature see the backup-hook feature in dar man page (-<, -> and -= options).


20 - Aborting an Operation

If the POSIX thread support is available, libdar will be built in a thread-safe manner, thus you may have several thread using libdar calls at the same time (but on different objects except concerning the libdar::statistics which can be shared between threads). You may then wish to interrupt a given thread. But aborting a thread form the outside (like sending it a KILL signal) will most of the time let some memory allocated or even worse can lead to dead-lock situation, when the killed thread was inside a critical section and had not got the opportunity to release a mutex. For that reason, libdar proposes a set of calls to abort any processing libdar call which is ran by a given thread.

   // next is the thread ID in which we want to have lidbar call canceled
   // here for simplicity we don't describe the way the ID has been obtained
   // but it could be for example the result of a call to pthread_self() as
   // defined in <pthread.h> system header file
pthread_t thread_id = 161720;
  
   // the most simple call is:
libdar::cancel_thread(thread_id);
   // this will make any libdar call in this thread be canceled immediately

   // but you can use something a bit more interesting:
libdar::cancel_thread(thread_id, false);
   // this second argument is true for immediate cancellation,
   // of false for a delayed cancellation, in which case libdar aborts the operation
   // but produces something usable. For example, if you were backing up something
   // you get a real usable archive which only contains files saved so far, in place
   // of having a broken archive which misses a catalogue at the end. Note that this
   // delayed cancellation needs a bit more time to complete, depending on the
   // size of the archive under process.

As seen above, cancellation can be very simple. What now succeeds when you ask for a cancellation this way? Well, an exception of type Ethread_cancel is thrown. All along his path, memory is released and mutex are freed. Last, the exception appears to the libdar caller. So, you can catch it to define a specific comportment. And if you don't want to use exceptions a special returned code is used.

try
{
    libdar::archive my_arch(...);
    ...
}
catch(libdar::Ethread_cancel & e)
{
    ... do something specific when thread has been canceled;
}

Some helper routines are available to know the cancellation status for a particular thread or to abort a cancellation process if it has not yet been engaged.

 pthread_t tid;
  
   // how to know if the thread tid is under cancellation process?
if(libdar::cancel_status(tid))
     cout << "thread cancellation is under progress for thread : " << tid << endl;
else
     cout << "no thread cancellation is under progress for thread : " << endl;

   // how to cancel a pending thread cancellation ?
if(libdar::cancel_clear(tid))
    cout << "pending thread cancellation has been reset, thread " << tid << " has not been canceled" << endl;
else
   cout << "too late, could not avoid thread cancellation for thread "<< tid << endl;

Last point, back to the Ethread_cancel exception, this class has two methods you may find useful, when you catch it:

try
{
   ... some libdar calls
}
catch(libdar::Ethread_cancel & e)
{
   if(e.immediate_cancel())
       cout << "cancel_thread() has been called with "true" as second argument" << endl;
   else
      cout << "cancel_thread() has been called with "false" as second argument" << endl;

   U64 flag = e.get_flag();
    ... do something with the flag variable...
}

    // what is this flag stored in this exception?
    // You must consider that the complete definition of cancel_thread() is the following:
    // void cancel_thread(pthread_t tid, bool immediate = true, U_64 flag = 0);
    // thus, any argument given in third is passed to the thrown Ethread_cancel exception,
    // value which can be retrieved thanks to its get_flag() method. The value given to this
    // flag is not used by libdar itself, it is a facility for user program to have the possibility
    // to include additional information about the thread cancellation.

    // supposing the thread cancellation has been invoked by:
libdar::cancel_thread(thread_id, true, 19);
   // then the flag variable in the catch() statement above would have received
   // the value 19.



21 - Dar_manager API


For more about dar_manager, please read the man page where are described in detail the available features. Note that for dar_manager there is not a "without exception" flavor, your program must be able to handle exceptions, which by the way are the same as the ones described above.

To get dar_manager features you need to use the class database which is defined in the libdar/database.hpp header file so you first need to include that file. Most of the methods of the database class do use options. For the same reason as previously seen for archive manipulation, these options are passed thanks to a container class. These container classes for options used by the database class are defined in the libdar/database_options.hpp file. Let's see the different method of the class database :

Database object construction

Two constructor are available. The first creates a brand-new but empty database in memory

database(const std::shared_ptr<user_interaction> & dialog);

As seen for libdar::archive dialog can be set to a null pointer if the default interaction mode (stdin/stdout/stderr) suits your need.
The second constructor opens an existing database from filesystem and stores its contents into memory ready for further use.

database(const std::shared_ptr<user_interaction> & dialog,
                 const std::string & base,
                 const database_open_options & opt);

  • dialog here to can be set to a null pointer or can point to an user_interaction object of your own
  • base is the path and filename of the database to read
  • opt is an object containing a few options. As seen with libdar::archive we can use an default temporary object to use default option

    database base;  
       // we have created an empty database (no archive in it) called "base"

    database other  = database(nullptr,
                               "/tmp/existing_base.dmd",
                               database_open_options());
      // we have created a database object called "other" which contains
      // (in RAM) all information that were contained in the
      // database file "/tmp/existing_base.dmd"
    

    database_open_option opt;
    opt.set_partial(true);
    opt.set_warn_order(false);
    database other2 = database(nullptr,
                               "/tmp/existing_base.dmd",
                               opt);
      // we have created yet another database object called "other2" which differs
      // from "other" by the option we used. While "other" is a fully loaded
      // database, "other2" is a partial database. This notion is explained
      // below

  • database_open_options::set_partial(bool value) leads dar to only load the database header into memory, which is quicker than loading the full database. But some operation we will see bellow need fully loaded database,  the other can work with both
  • database_open_options::set_partial_read_only(bool value) in addition to have only the header the archive is open in read-only mode which of course forbids any modification to the database but is even faster than just a partial read-write database. For just database listing this is perfectly adapted.
  • database_open_options::set_warn_order(bool value) avoid warning about ordering problem between archive
In the following we will indicate whether a database operation can be applied to a partially loaded database or not. All operation can be applied to a fully loaded databse.

Database's methods

First we will see methods that can work with a partial and read-only database
  • show_contents() : list the archives used to build the database
  • get_options() : list the options that will be passed to dar (as defined with the set_options() method)
  • get_dar_path() : return the path to dar (or empty string if relying on the PATH variable)
Second we will see methods that can work with  partially loaded databases:
  • all methods seen above
  • dump(...) : it is used to write back the database to a file.
  • change_name() : change the basename of the archive which index is given in argument
  • set_path() : change the path to the archive which index is given in argument
  • set_options() : change the default options to always pass to dar when performing restoration
  • set_dar_path() : specify the path to dar (use empty string to rely on the PATH variable)
last, let's see the database methods that can work with completely loaded databases:
  • all methods seen above
  • add_archive() : add an archive to the database
  • remove_archive() : remove an archive from the database
  • set_permutation() : change archive relative order within the database
  • show_files() : list the files which are present in the given archive
  • show_version() : list the archive where the given file is saved
  • show_most_recent_stats() :  compute statistics about the location of most recent file versions
  • restore() : restore a set of given files given in argument.
Well, you might now say that as description this is a bit light for a tutorial, yes. In fact these call are really very simple to use, you can find a complete description in the API reference documentation. This documentation is built if doxygen is available and is put under doc/html after calling make in the source package. It is also available from dar's homepage.


22 - dar_slave API

dar_slave role is to read an archive while interacting with a dar process through a pair of pipes. Dar asks portion of the archive or information about the archive in the first pipe from dar to dar_slave. And dar_slave sends the requested information into the other pipe toward dar.

Since API 6.0.x, dar_slave has an API. It is implemented by the class libdar::libdar_slave. You need firs to create an object using the following constructor:

libdar_slave(std::shared_ptr<user_interaction> & dialog,
             const std::string & folder,
             const std::string & basename,
             const std::string & extension,
             bool input_pipe_is_fd,
             const std::string & input_pipe,
             bool output_pipe_is_fd,
             const std::string & output_pipe,
             const std::string & execute,
             const infinint & min_digits);

  • dialog as seen for other libdar classes can be set to a null pointer for interaction on stdin and stdout
  • folder is the directory where resides the archive to read
  • basename is the basename of the archive
  • extension should always be set to "dar"
  • input_pipe_is_fd if set to true, the next argument is not the path to a named pipe but a number corresponding to a file descriptor open open in read mode
  • input_pipe is the path of a named pipe to read from. It can also be an empty string to use stdin as input pipe
  • out_pipe_is_fd if set to true, the next argument is not the path to a named pipe but a number corresponding to a file descriptor open in write mode
  • output_pipe is the path of a named pipe to write from. It can also be an empty string to use stdout as input pipe
Once the object is created, you will need to call the libdar_slave::run() method which will end when the dar process at the other end will no more need of this slave

libdar::libdar_slave slave(nullptr,
                           "/tmp",
                           "first_backup",
                           "dar",
                           false,
                           "/tmp/toslave", // assuming this is an existing named pipe
                           false,
                           "/tmp/todar",   // assuming this is also an existing named pipe
                           "echo 'reading slice %p/%b.%N.%e in context %c'",
                           0);

slave.run();
 
  // once run() has returned, you can launch it again for another process or the same
  // one as previously has access to the /tmp/first_backup.*.dar archive


23 - dar_xform API

dar_xform creates a copy of a given archive modifying its slicing. it does not require decompressing nor deciphering the archive to do so. There is different constructor depending whether the archive is read from filesystem, from a named pipe of from a provided file descriptor

Reading from a file

libdar_xform(const std::shared_ptr<user_interaction> & ui,
             const std::string & chem,
             const std::string & basename,
             const std::string & extension,
             const infinint & min_digits,
             const std::string & execute);
  • ui as seen for other libdar classes can be set to a null pointer for interaction on stdin and stdout
  • chem is the directory where resides the archive to read
  • basename is the basename of the archive
  • extension should always be set to "dar"
  • min_digits is the minimum number of digits slice number in filename have been created with (use zero if you don't know what it is)
Reading from a named pipe

libdar_xform(const std::shared_ptr<user_interaction> & dialog,
             const std::string & pipename);
  • dialog as seen for other libdar classes, it can be set to nullptr
  • pipename complete path to the named pipe to read the archive from
Reading from a file descriptor

libdar_xform(const std::shared_ptr<user_interaction> & dialog,
             int filedescriptor);
  • dialog same as above
  • filedescriptor is an read opened file descriptor to read the archive from
Once the libdar::libdar_xform object is created it can copy the referred archive to another location in another form thanks to one of the two libdar_xform::xform_to methods. There is not link between the constructor used and the libdar_xform::xform_to flavor used, any combination is possible.

Creating a single or multi-sliced archive on filesystem
void xform_to(const std::string & path,
              const std::string & basename,
              const std::string & extension,
              bool allow_over,
              bool warn_over,
              const infinint & pause,
              const infinint & first_slice_size,
              const infinint & slice_size,
              const std::string & slice_perm,
              const std::string & slice_user,
              const std::string & slice_group,
              hash_algo hash,
              const infinint & min_digits,
              const std::string & execute);

Creating a single sliced archive toward a filedescriptor

void xform_to(int filedescriptor,
              const std::string & execute);

Here follows an example of use. We will convert a possibly multi-sliced archive to a single slice one generating a sha512 hash file on-fly

libdar::libdar_xform transform(nullptr,
                               "/tmp",
                               "my_first_archive",
                               "dar",
                               0,
                               "echo 'reading slice %p/%b.%N.%e context is %c'");

transform.xform_to("/tmp",
                   "my_other_first_archive",
                   "dar",
                   false,     // no overwriting allowed
                   true,      // does not matter whether we warn or not as we do not allow overwriting
                   0,         // no pause between slices
                   0,         // no specific first slice
                   0,         // no slicing at all (previous argument is thus not used anyway)
                   "",        // using default permission for created slices
                   "",        // using default user ownership for created slices
                   "",        // using default group ownership for created slices
                   libdar::hash_algo::sha512, // the hash algo to use (for no hashing use hash_none instead)
                   0,         // min_digits ... not using this feature here
                   "echo 'Slice %p/%b.%N.%e has been written. Context is %c'");



24 - Compilation & Linking

Compilation

All the symbols found in the libdar API except the one relative to dar_manager (see below) are defined from <dar/libdar.h>. So you should only need to include this header. If the header file is not located in a standard directory, in order to compile your code, you may need some extra flags to pass to the compiler (like -I/opt/...). The pkg-config tool can help here to avoid system dependent invocation:


> cat my_prog.cpp

#include <dar/libdar.h>

main()
{
   libdar::get_version(...);
   ...
}

> gcc `pkg-config --cflags libdar` -c my_prog.cpp


Linking


Of course, you need to link your program with libdar. This is done by adding -ldar plus other library libdar can rely on like libz, libbzip2, liblzo or libgcrypt, depending on the feature activated at compilation time. Here too, pkg-config can provide a great help to avoid having system dependent invocation:


> gcc pkg-confg --libs libdar` my_prog.o -o my_prog

Libdar's different flavors


Well, all the compilation and linking steps described above assume you have a "full" libdar library. Beside the full (alias infinint) libdar flavor, libdar also comes in 32 and 64 bits versions. In these last ones, in place of internally relying on a special type (which is a C++ class called infinint) to handle arbitrary large integers, libdar32 relies on 32 bits integers and libdar64 relies on 64 bits integers (there are limitations which are described in doc/LIMITATIONS). But all these libdar version (infinint, 32bits, 64bits) have the same interface and must be used the same way, except for compilation and linking.

These different libdar versions can coexist on the same system, they share the same include files. But the LIBDAR_MODE macro must be set to 32 or 64 when compiling or linking with libdar32 or libdar64 respectively. The LIBDAR_MODE macro defines the way the "class infinint" type is implemented in libdar, and thus changes the way the libdar headers files are interpreted by the compiler. pkg-config --cflags will set the correct LIBDAR_MODE, so you should only bother calling it with either libdar, libdar32 or libdar64 depending on your need : "pkg-confg --cflags libdar64" for example.

> cat my_prog.cpp
#include <dar/libdar.h>

main()
{
   libdar::get_version(...);
   ...
}
> gcc -c `pkg-config --cflags libdar32` my_prog.cpp


> gcc `pkg-config --libs libdar32` my_prog.o -o my_prog

and replace 32 by 64 to link with libdar64.