Inifile

Basics

The other day I simply couldn't resist the idea to play around with ini files. What you see below is a transformation of ini files to xml and from xml format. In between probably some hashes will appear. This code can read very simple ini files and then print them in xml format. Yes I'm aware that this was already CPANned to death (courtesy of Randal L. Schwartz at least where I've seen it first) See these links also for some inspiration:

The main goal was achieved. I have now means to transform the conf/ini files to xml. Then It is possible to do all other conversions using xsl which is very convenient. So basically it is enough to write "importers/readers" for specific inifile-like data which save in xml format. Then onwards from that point you are all set with the standard tools. Now to find something that transforms xsd files into XForms. Then XForms using AJAXForms to create html/java input and the perfect tool for webconfig is born (Yes I know Webmin). Ok so case closed for now.

XML

Basic

As a first step I have created one sample xml file. This is how I imagine that the ini file should look after converting it to xml. The tag inifile should indicate that the data represent a simple ini file. The src attribute represents the location of the ini file. Maybe in future it would be desirable to have something like file:/ /poseidon/etc/inifile.ini . Something like that would be also interesting for the crontab project of mine of course. The inifile itself then contains at least one section which can contain values then. The section has a attribute caled name which indicates the section name. The value has also a attribute which is called name. The value itself is then stored between the value tags. You can see the xml file below. Because of testing I have included an empty section as well as an empty value

<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE inifile SYSTEM "inifile.dtd">
<inifile src="test.xml">
    <section name="special">
        <value name="lusuwa">&quot;Whatever You May Think&quot;</value>
        <value name="zhwado">45-13,87,papaya</value>
    </section>
    <section name="common">
        <value name="empwty"/>
        <value name="ogwana">45</value>
        <value name="bwanga">ba,we,gt,su</value>
    </section>
    <section name="empty"/>
</inifile>

Extended

The next snipet is a already converted mt-daapd.conf file. This is how it will work for me. The comments are stored as text value for the given section or entry. So the value is moved to an attribute called value thus I decided to rename the value tag to entry. New lines are preserved and the comment tag is also preserved. To add the comment tag by means of the xsl is really a pain in the ass. So I'm glad I have to do that ugly xsl thing for new lines only in the html conversion to add in the <br/> tag.

<?xml version="1.0" encoding="UTF-8"?>
<inifile src="mtdaapd.conf">
#
# some comments at the end, well actually they will end up on top anyway
#
 
<section name="general"># $Id: mt-daapd.conf.templ 1412 2006-10-25 02:55:15Z rpedde $
#
# This is the mt-daapd config file.
#
# If you have problems or questions with the format of this file,
# direct your questions to rpedde@users.sourceforge.net.
#
# You can also check the website at http://mt-daapd.sourceforge.net,
# as there is a growing documentation library there, peer-supported
# forums and possibly more.
#
 
<entry name="admin_pw" value="mt-daapd">
#
# admin_pw (required)
#
# This is the password to the administrative pages
#
 
</entry>
<entry name="always_scan" value="0">
# always_scan
#
# The default behavior is not not do background rescans of the
# filesystem unless there are clients connected.  The thought is to
# allow the drives to spin down unless they are in use.  This might be
# of more importance in IDE drives that aren\'t designed to be run
# 24x7.  Forcing a scan through the web interface will always work
# though, even if no users are connected.
 
</entry>
</section>

DTD

Basic

The next step was to create the dtd for validating my xml files. It is pretty simple. The root element is inifile. It must contain at least one section element. Well there is no particular reason for that. You can change it any time by replacing the + sign with a *. The section can contain values, but doesn't have to. Value is some parseable data. The src can be present but don't have to. and the name attribute is required for section and value

<!ELEMENT inifile (section+)>
<!ELEMENT section (value*)>
<!ELEMENT value (#PCDATA)>
<!ATTLIST inifile src CDATA #IMPLIED>
<!ATTLIST section name CDATA #REQUIRED>
<!ATTLIST value name CDATA #REQUIRED>

INI

Now what you see is simply the transcript of the xml into ini format. At least how I would like to have it.

[special]
lusuwa="Whatever You May Think"
zhwado=45-13,87,papaya
[common]
empwty=
ogwana=45
bwanga=ba,we,gt,su
[empty]

XSL

INI

Basic XML

Below you can see a xsl stylesheet to print the xml file as a regular inifile. This means you can convert any ini file in the xml form into a plain textual ini file.

<?xml version="1.0" encoding="UTF-8"?>
<xsl:stylesheet version="1.0" xmlns:xsl="http://www.w3.org/1999/XSL/Transform">
 
<xsl:output method="text"/>
<xsl:strip-space elements="*"/>
 
<xsl:template match="/">
    <xsl:apply-templates/>
</xsl:template>
 
<xsl:template match="inifile/section">
    <xsl:text>[</xsl:text>
    <xsl:value-of select="@name"/>
    <xsl:text>]
</xsl:text>
    <xsl:for-each select="value">
      <xsl:value-of select="@name"/>
      <xsl:text>=</xsl:text>
      <xsl:value-of select="."/>
      <xsl:text>
</xsl:text>
        </xsl:for-each>  
</xsl:template>
 
</xsl:stylesheet>

Extended XML

Ok, here is the stylesheet which I use to convert the "extended" XML format to the INI format. Nothing special, just the handling of the renamed tags and attributes was added. So not a major change.

<?xml version="1.0" encoding="UTF-8"?>
<xsl:stylesheet version="1.0" xmlns:xsl="http://www.w3.org/1999/XSL/Transform">
 
<xsl:output method="text"/>
<xsl:strip-space elements="*"/>
 
<xsl:template match="/">
        <xsl:apply-templates/>
</xsl:template>
 
<xsl:template match="inifile/section">
        <xsl:value-of select="text()"/>
        <xsl:text>[</xsl:text>
    <xsl:value-of select="@name"/>
        <xsl:text>]
</xsl:text>
        <xsl:for-each select="entry">
          <xsl:value-of select="text()"/>
      <xsl:value-of select="@name"/>
          <xsl:text>=</xsl:text>
      <xsl:value-of select="@value"/>
          <xsl:text>
</xsl:text>
    </xsl:for-each>
</xsl:template>
 
</xsl:stylesheet>

HTML

Basic XML

Additionally there is a stylesheet which was created as a side product to convert the xml inifile into a html table for convenient viewing in a web browser.

<?xml version="1.0" encoding="UTF-8"?>
<xsl:stylesheet version="1.0" xmlns:xsl="http://www.w3.org/1999/XSL/Transform">
 
<xsl:template match="/">
  <html>
  <body>
    <h2><xsl:value-of select="inifile/@src"/></h2>
    <table border="1">
    <xsl:apply-templates/>
    </table>
  </body>
  </html>
</xsl:template>
 
<xsl:template match="inifile/section">
    <tr bgcolor="#aaaaaa">
      <td colspan="2"><b><xsl:value-of select="@name"/></b></td>
    </tr>
    <xsl:for-each select="value">
    <tr>
      <td><xsl:value-of select="@name"/></td>
      <td><i><xsl:value-of select="."/></i></td>
    </tr>
    </xsl:for-each>  
</xsl:template>
 
</xsl:stylesheet>

Extended XML

Well this was a real pain. But thanks to "google knows everything" it had a happy end. This transformation of new lines onto a <br/> is really neat. Beats me why it is so and I find it a bit unconvenient. Well another "problem" with this xsl is that if I have som text in the root tag inifile it appears right on top of my nice table . And that I find mean! Explanations are welcome. The text in the inifile appears when you append something to the conf/ini file as comment at the end.

<?xml version="1.0" encoding="UTF-8"?>
<xsl:stylesheet version="1.0" xmlns:xsl="http://www.w3.org/1999/XSL/Transform">
 
<xsl:template match="/">
  <html>
  <body>
    <h2><xsl:value-of select="inifile/@src"/></h2>
    <table border="1">
        <xsl:apply-templates/>
    </table>
  </body>
  </html>
</xsl:template>
 
<xsl:template name="hash">
        <xsl:param name="text" select="text()"/>
        <xsl:value-of select="translate($text,'#','')"/>
</xsl:template>
 
<xsl:template name="break">
        <xsl:param name="text" select="text()"/>
        <xsl:choose>
        <xsl:when test="contains($text, '&#xa;')">
                <xsl:call-template name="hash">
                        <xsl:with-param name="text" select="substring-before($text, '&#xa;')"/>
                </xsl:call-template>
                <br/>
                <xsl:call-template name="break">
                        <xsl:with-param name="text" select="substring-after($text,'&#xa;')"/>
                </xsl:call-template>
        </xsl:when>
                <xsl:otherwise>
                <xsl:call-template name="hash">
                        <xsl:with-param name="text" select="$text"/>
                </xsl:call-template>
                </xsl:otherwise>
        </xsl:choose>
</xsl:template>
 
<xsl:template match="inifile/section">
    <tr bgcolor="#aaaaaa">
      <td><b><xsl:value-of select="@name"/></b></td>
          <td colspan="2"><xsl:call-template name="break"/></td>
    </tr>
        <xsl:for-each select="entry">
    <tr>
      <td><xsl:value-of select="@name"/></td>
      <td><i><xsl:value-of select="@value"/></i></td>
      <td><xsl:call-template name="break"/></td>
    </tr>
    </xsl:for-each>
</xsl:template>
 
</xsl:stylesheet>

Transform!

You have to use either the xsltproc to transform the xml file into the ini or html file. Or you can specify the line below in the xml inifile to use the xsl either for textual representation or html. (the name of the xsl file must go in here. I have used the inifile2text.xsl for the ini file transformation and the inifile2html.xsl for the html transformation)

<?xml version="1.0" encoding="UTF-8"?>
<?xml-stylesheet type="text/xsl" href="inifile2text.xsl"?>
...

Perl

For no particular reason I have started to code this as "object" in perl. Let's see how it ends.

Test data

Ok so now on to some coding. First I have created a hash which should be my internal representation of the xml/ini file. Nothing very special, but I can use it for testing.

my %example_structure=(
    common => { 
        ogwana => '45',
        bwanga => 'ba,we,gt,su',
        empwty => '',
    },
    special => { 
        zhwado => '45-13,87,papaya',
        lusuwa => '"Whatever You May Think"',
    },
    empty => {
    },
);

Create object

At the begining there is new. And some test data loading just to be sure. The new method will create an empty hash where we later store our data. The ini_test method will simply move our test data into the hash. Here I have actually created two key in the basic hash. One is called info and actually holds the value for the src attribute of the inifile root node. The second key is called data and stores the actual inifile data. The keys of this hash are actually the names used in the section tag. Whereas every section has again a hash inside itself where the keys represent the name attribute of the value tag and the value itself is stored under this key. Simple.

# create an empty hash
sub new ($)
{
my $type=shift;
my $self={};
 
    bless ($self, $type);
    return $self;
}
 
sub ini_test ($$)
{
my $self     =shift;
my $file_name=shift;
 
    $self->{'info'} = $file_name;
    $self->{'data'} = { %example_structure };
}

Later on we will use also the XML::Simple module to read and write our files. In those cases so that we don't have to create the object every time we precreate one in the new subroutine. The root node name is set to inifile

sub new ($)
{
my $type=shift;
my $self={};
 
    bless ($self, $type);
    my $xml = new XML::Simple(RootName=>'inifile');
    $self->{'xml'}=$xml;
    return $self;
}

Load INI file

This method will load the textual ini file and convert it into a simple hash which I have created for testing purposes.

Basic version

This only loads the values and stores everything in the hash. The empty section is created with a empty hash. The question is if it is feasible to create it with undef perhaps.

sub ini_load ($$)
{
my $self     =shift;
my $file_name=shift;
my $section  ="_";
 
    $self->{'info'} = $file_name;
    open (INIFILE, $file_name);
    while (my $line = <INIFILE>) 
    {
        if ( $line=~m/^\[(.*?)\]/ )
        {
            $section=$1;
            $self->{'data'}{$section}={}
        }
        elsif ( $line=~m/^(.*?)=(.*)$/ )
        {
            $self->{'data'}{$section}{$1}=$2;
        }
    }
    close(INIFILE);
}

Callback version

Well for some funny reason I have decided to incorporate the possiblity of invoking a user defined function for every value which is to be stored in the hash. So with this change you get the chance to preprocess the value befor it's going to be stored. You need to add a new input parameter which is a reference to the callback function and invoke it on the right place.

sub ini_load ($$$)
{
...
my $preproces=shift;
...
        elsif ( $line=~m/^(.*?)=(.*)$/ )
        {
            my $the_key=$1;
            my $the_val=$2;
            $self->{'data'}{$section}{$the_key}=$preproces->($the_val);
        }
...
}

And for your convenience I have also created a simple callback function. This function examines the string for commas. If there are commas in the string then the string will be splitted up into an array. So at the end instead of a single string value you have a array hidden under it. No idea if its usable.

sub main::to_array ($)
{
my $data=shift;
 
    if ( $data =~ m/,/ )
    {
        my @new_data=split ( /,/, $data );
        return \@new_data;
    }
    return $data;
}

And in order to test it properly you can execute this to see the results

my $test_inifile=inifile->new();
$test_inifile->ini_load("test.ini",\&main::to_array);

So after running the code the output from the test data which you get (use the Data::Dumper module) look like this

$VAR1 = {
          'special' => {
                         'lusuwa' => '"Whatever You May Think"',
                         'zhwado' => [
                                       '45-13',
                                       '87',
                                       'papaya'
                                     ]
                       },
          'empty' => {},
          'common' => {
                        'empwty' => '',
                        'ogwana' => '45',
                        'bwanga' => [
                                      'ba',
                                      'we',
                                      'gt',
                                      'su'
                                    ]
                      }
        };

XML::Simple format basic

This version of the load subroutine saves the ini file into a hash which conforms with the XML::Simple format. You have to change only three lines of code. See more on the XML::Simple topic in the Save XML with XML::Simple chapter.

...
    $self->{'data'}{'src'} = $file_name;
...
        if ( $line=~m/^\[(.*?)\]/ )
        {
            $section=$1;
            $self->{'data'}{'section'}{$section}={};
        }
        elsif ( $line=~m/^(.*?)=(.*)$/ )
        {
            $self->{'data'}{'section'}{$section}{'value'}{$1}{'content'}=$2;
        }
...

XML::Simple format extended

And here you have the load ini file version which will preserve also the comments in the conf/inifile. It is also adapted to the new tag and attribute structure which was used for the extended xml format.

sub ini3xml ($$)
{
my $self     =shift;
my $file_name=shift;
my $section  ="_";
my $comment;
my $i=0;
 
        $self->{'data'}{'src'} = $file_name;
        open (INIFILE, $file_name);
        while (my $line = <INIFILE>) {
                $i++;
                # remove leading and trailing whitespace
                $line=trim($line);
                if ( $line=~m/^$/ )
                {
                        # preserve new lines
                        $comment.="\n";
                }
                elsif ( $line=~m/^#(.*)/ )
                {
                        # the comment sign shoud be added
                        $comment.="#".$1."\n";
                }
                elsif ( $line=~m/^\[(.*?)\]/ )
                {
                        $section=$1;
                        # remove leading and trailing whitespace
                        $section=trim($section);
                        $self->{'data'}{'section'}{$section}{'content'}=$comment;
                        $comment='';
                }
                elsif ( $line=~m/^(.*?)=(.*)$/ )
                {
                        my $attribute=$1;
                        my $value=$2;
                        # remove leading and trailing whitespace
                        $attribute=trim($attribute);
                        $value=trim($value);
                        $self->{'data'}{'section'}{$section}{'entry'}{$attribute}{'content'}=$comment;
                        $self->{'data'}{'section'}{$section}{'entry'}{$attribute}{'value'}=$value;
                        $comment='';
                }
        }
        # flush the rest of comments at the end of the file
        $self->{'data'}{'content'}=$comment;
        close(INIFILE);
}

Save as XML

The save has three versions. One is the simple and the second version also uses the callback to change the array back to string. The third directly utilizes the XMLout method of XML simple and is the most convenient. Well but first you have to get your hash into a form XML::Simple convert correctly.

Basic version

sub xml_save ($$)
{
my $self     =shift;
my $file_name=shift;
 
    if ( length($file_name) eq 0 )
    {
        $file_name=$self->{'info'};
    }
    open (XMLFILE, ">$file_name");
    print XMLFILE "<?xml version=\"1.0\" encoding=\"UTF-8\"?>\n";
    print XMLFILE "<inifile src=\"".$file_name."\">\n";
    foreach my $section ( keys %{$self->{'data'}} )
    {
        print XMLFILE "\t<section name=\"".$section."\"";
        if ( scalar(keys %{$self->{'data'}{$section}}) ne 0 )
        {
            print XMLFILE ">\n";
            foreach my $value ( keys %{$self->{'data'}{$section}} )
            {
                print XMLFILE "\t\t<value name=\"".$value."\"";
                if ( length($self->{'data'}{$section}{$value}) ne 0 )
                {
                    print XMLFILE ">".$self->{'data'}{$section}{$value}."</value>\n";
                }
                else
                {
                    print XMLFILE "/>\n";
                }
            }
            print XMLFILE "\t</section>\n";
        }    
        else
        {
            print XMLFILE "/>\n";
        }
    }
    print XMLFILE "</inifile>\n";
    close(XMLFILE);
}

Callback version

This is the function which uses the callback to shring the array into a string. Well actually the callback could do anything. You need to add the reference to the callback subroutine and invoke it.

sub xml_save ($$$)
{
...
my $preproces=shift;
... 
        if ( length($self->{'data'}{$section}{$value}) ne 0 )
        {
            my $save_val=$preproces->($self->{'data'}{$section}{$value});
            print XMLFILE ">".$save_val."</value>\n";
        }
        else
              {
                       print XMLFILE "/>\n";
               }
...
}

And of course the callback function which will change the array into a string

sub main::to_string ($)
{
my $data=shift;
 
    if ( ref($data) eq 'ARRAY' )
    {
        my $string="";
        foreach ( @{ $data } )
        {
            $string.=$_.",";
        }
        chop($string);
        return $string;
    }
    return $data;
}

Now only the invocation is left over. Here you can see how to call the function with the callback reference.

$test_inifile->xml_save("test.xml",\&main::to_string);

Now it comes to me I could use this callback to convert those things like quotes, apostrophes etc. with this callback thingie. Hey, it turned out useful.

XML::Simple format

If you try to process our hash with the XML::Simple module then the generated xml file look like this

<opt>
  <anon>common</anon>
  <anon bwanga="ba,we,gt,su" empwty="" ogwana="45" />
  <anon>special</anon>
  <anon lusuwa="&quot;Whatever You May Think&quot;" zhwado="45-13,87,papaya" />
</opt>

which is not exactly what we are looking for. Look in the Load INI file for a version of ini_load subroutine which can load the ini file into a has conforming with the XML::Simple format. So if we have that we can save the file using the XML::Simple module.

sub xml_save ($$)
{
my $self     =shift;
my $file_name=shift;
 
    my $data=$self->{'xml'}->XMLout($self->{'data'});
}

The code only prints the output into a variable. No actual writing to file occures, yet. I will maybe add it later. The good thing is also that all disallowed characters are automatically replaced by the corresponding entities. Anyhow below you can see how to do this withoud the XML::Simple module.

Replace entities

In xml there is a restriction on characters which can't be used in the value between the tags. You can read about it in the section entities. The shortcoming of this approach is of course if you use another entities like &nbps; that the ampersand & will also be replaced. The solution might be to enter all entities into the entity_map.

my %entity_map=(
    '&quot;' => '"',
    '&lt;'   => '<',
    '&gt;'   => '>',
    '&apos;' => "'",
);
 
sub main::unpack_entity ($)
{
my $data=shift;
 
    foreach ( keys %entity_map )
    {
        $data =~ s/$_/$entity_map{$_}/g;
    }
    $data =~ s/&amp;/&/g;
    return $data;
}
 
sub main::pack_entity ($)
{
my $data=shift;
 
    $data =~ s/&/&amp;/g;
    foreach ( keys %entity_map )
    {
        $data =~ s/$entity_map{$_}/$_/g;
    }
    return $data;
}

Next question is that maybe instead of specifing the ampersand and dot-comma in the entity_map this could be used directly in the substitution. I haven't tested this code yet. So you have to try for yourself.

my %entity_map=(
    'quot'  => '"',
    'lt'      => '<',
    'gt'     => '>',
    'apos' => "'",
);
 
...
        $data =~ s/&$_;/$entity_map{$_}/g;
...
        $data =~ s/$entity_map{$_}/&$_;/g;
...

Save as INI

Basic version

This code saves the original hash into a file. For the hash format of XML::Simple see further on.

sub ini_save ($$)
{
my $self     =shift;
my $file_name=shift;
 
    if ( length($file_name) eq 0 )
    {
        $file_name=$self->{'info'};
    }
    open (INIFILE, ">$file_name");
    print INIFILE "# ".$file_name."\n";
    foreach my $section ( keys %{$self->{'data'}} )
    {
        print INIFILE "[".$section."]\n";
        foreach my $value ( keys %{$self->{'data'}{$section}} )
        {
            print INIFILE $value."=".$self->{'data'}{$section}{$value}."\n";
        }
    }
    close (INIFILE);
}

Run the code

my $test_inifile=inifile->new();
$test_inifile->ini_test("test.ini");
$test_inifile->xml_save("");
$test_inifile->ini_save("");

Load XML with XML::Simple

Ok, now the saved XML file can be read using the XML::Simple module. The subroutine would look like this:

sub xml_load ($$)
{
my $self     =shift;
my $file_name=shift;
 
    my $xml = new XML::Simple;
    my $data = $xml->XMLin($file_name);
}

And the output from the data dumper look like this (see below). And yes I have added one section with quotes, apostrophes and whatnot to test the entiti conversion. But what is more disturbing is that the darn hash doesn't look like the one we used to have. This is influenced also by parameters you use for the new method of the XML::Simple. Let's just say I had my share of fun with the hash format and how XML::Simple understood my poor tries to get to write the xml file I want.

$VAR1 = {
          'src' => 'test.xml',
          'section' => {
                         'special' => {
                                        'value' => {
                                                     'lusuwa' => {
                                                                   'content' => '"Whatever You May Think"'
                                                                 },
                                                     'zhwado' => {
                                                                   'content' => '45-13,87,papaya'
                                                                 }
                                                   }
                                      },
                         'entiting' => {
                                         'value' => {
                                                      'apostrophes' => {
                                                                         'content' => '\'\'\'\''
                                                                       },
                                                      'quotes' => {
                                                                    'content' => '"""'
                                                                  },
                                                      'ampersand' => {
                                                                       'content' => '&&&&&'
                                                                     },
                                                      'smaller' => {
                                                                     'content' => '<'
                                                                   },
                                                      'greater' => {
                                                                     'content' => '>>'
                                                                   }
                                                    }
                                       },
                         'common' => {
                                       'value' => {
                                                    'empwty' => {},
                                                    'ogwana' => {
                                                                  'content' => '45'
                                                                },
                                                    'bwanga' => {
                                                                  'content' => 'ba,we,gt,su'
                                                                }
                                                  }
                                     },
                         'empty' => {}
                       }
        };

ToDo's and Comments

  • DONE adding a dtd could be interesting
  • DONE It won't properly format the section if the section is empty (well it is properly closed but could be shortened like the empty value)
  • DONE (it writes to file) it says save but it just prints out
  • DONE it doesn't properly converts quotes, apostrophes, greater than and lower than signs.
  • DONE (html as bonus :-) well a xsl to convert the xml into ini would be interesting ?
  • DONE (can be done better the standard way) I hope you don't expect me to write a xml_load function
  • DITCHED This is an interesting ini file definition. Maybe I stick to it. Just to have some standard.
  • DITCHED (attribute exists fill it however you want :-) adding some kind of resource locator ? URL/URI/IRI whatever ? (also an idea for the crontab maybe) could be something like file://poseidon/etc/crontab etc.
  • DONE (comments stored as text for the given node) adding the extended tag to store comments or whatever
  • DITCHED adding some posibility to keep line numbers like in crontab
  • DITCHED (It's a mess anyway) Make a decent structure for the text like Load, Save, XML, INI etc.
  • PHP!
Add a New Comment
Unless otherwise stated, the content of this page is licensed under Creative Commons Attribution-Share Alike 2.5 License.