Merging associative arrays in PHP

It’s nice when serendipity is your friend! I was porting my Bitly class from Pike to PHP – I know there’s probably a hundred PHP classes already out there, but mine is better coded ;) – and noted by accident that I had used some Pike syntax in my PHP class but it was working anyway. So what was I doing? In Pike there’s separate data type for associative arrays called mapping. In Pike, in general, when merging two objects you just join them with a + sign. Thus merging two mappings you do like

9 lines of Pike
  1. mapping m1 = ([ “key1” : “Value 1”, “key2” : “Value 2” ]);
  2. mapping m2 = ([ “key3” : “Value 3” ]);
  3. write(“My mapping: %O\n”, m1 + m2);
  4. //> My mapping: ([ /* 3 elements */
  5. //> “key1”: “Value 1”,
  6. //> “key2”: “Value 2”,
  7. //> “key3”: “Value 3”
  8. //> ])

And I noted that I had done the same thing in PHP and the result was perfectly valid:

11 lines of PHP
  1. $a1 = array(“key1” => “Value 1”, “key2” => “Value 2”);
  2. $a2 = array(“key3” => “Value 3”);
  3. echo “My mapping: “;
  4. print_r($a1 + $a2);
  5. //> My mapping: Array
  6. //> (
  7. //> [key1] => Value 1
  8. //> [key2] => Value 2
  9. //> [key3] => Value 3
  10. //> )

This method doesn’t work on flat array though so there you’ll still have to use array_merge(), but pretty nice anyway.

And oh, the PHP Bitly class will be part of the new PLib release once done!

UTF-8 encoding/decoding in C

I was working on a simple database to Excel XML exporter the other day and decided to write it in C. Now, the problem was that since the Swedish language contains non-ascii characters the output needs to be UTF-8 encoded. C doesn’t have a built-in function for this – it seems I should add since I’m a C rookie – and no matter how I searched at Google I couldn’t find anything useful. So I thought…

Look at PHP

…why not look at the source code of PHP and see how the PHP functions utf8_encode and utf8_decode are being done. So I downloaded the source of PHP and with a little find . -name *.c -print | xargs grep "utf8_encode" I found the functions in xml.c. Thankfully they weren’t too complicated – when dug out from the rest of the XML functions – so I didn’t take too long before I had them as standalone functions.

This is how they are used:

12 lines of C/C++
  1. #include “utf8.h”
  2. int main(int argc, char **argv)
  3. {
  4. char *iso_str = “Pontus Östlund”;
  5. char *utf8_str;
  6. utf8_str = utf8_encode(iso_str);
  7. iso_str = utf8_decode(utf8_str);
  8. return 0;
  9. }

And it seems to be working quite OK!

Sources at Github

Roxen Application Launcher 0.3

I just created an updated version of Roxen Launcher. I got an error report from one who tried to build the application:

./ApplCon.cs(232,24): error CS0122:
‘System.Net.Sockets.TcpListener.Server’ is inaccessible due
to its protection level

An explanation of the error can be found at MSDN. The reason seems to be due to TcpListener.Server being a protected property but I tried to access it as public. The strange thing is that I didn’t get an error about it nor on my machine at home or at work. Now I created a derived class of TcpListener.Server so that I can access the protected property.

We’ll see if it works!

Roxen Application Launcher 17:31, Sat 17 October 2009 :: 97.3 kB

Gnome Font Manager

One thing I miss on Linux Gnome is font manager. Not just a font viewer but a proper manager like the old Adobe Type Manager. So I thought: Well, lets create one then! It might be that it already exist some font managers for Linux/Gnome but as always; this will be a good project for learning new stuff so I really don’t care if there are 1000 font managers out there ;)

Font parsing

The first thing to do, and that I have done, is porting my font parser from PLib to C#. That was no major head ache. There are at least to different font classes available in C# Mono (`System.Drawing.Font` and `Pango.Font`) but they don’t give all information about the font that I want.

Worth mentioning is that I heavily used the Mono DataConverter class to unpack the binary strings in the fonts. The unpack() function in PHP is just tremendous and there doesn’t seem to be a native alike in C#. But thanks to Miguel de Icasa’s DataConverter it went quite alright.

Font preview

The next thing to do was figuring out how to create the font previews. And I figured it out ;) First I though of using the console program gnome-thumbnail-font to create the previews but I had to throw that one into the bin since it doesn’t seem to handle multi line text. Since I’ve never used the graphics functions in C# before I came to the conclusion that I had to create the previews all by my self. It was quite easy finding good examples on the net of how to create text images with C#. A couple of hours later that problem was also solved (as you can see in the screen shot below). And man the graphics stuff in C# is fast. The preview images are generated instantly!

Next step

I have a lot left to do before this is a useful program but we’re heading there. One feature I’m planning on implementing is the ability to create your own font sets that you can activate/deactivate.

And I will probably come up with some more stuff to add, but that will be a later head ache!

Screen shot: Gnome Font Manager
Gnome Font Manager

Roxen Application Launcher for Linux

I thought I should broaden my C# knowledge a bit and you know how it is: To learn new stuff you need a real project to work on or else you will lose the fire sooner than later. So I came up with a good project that is actually useful to me: Porting Roxen’s “Application Launcher” to C#. There’s nothing wrong with the original one, written in Pike, except that it uses GTK 1 which is quite hideous (in an aesthetic meaning) compared to the newer GTK 2. And I also though it would be cool to create a panel applet (in the notification area of Gnome so you could put the Application Launcher in the background).

BTW: For those of you not knowing what the heck Roxen’s Application Launcher (AL here after) is here’s a brief explanation: Files in Roxen CMS is stored in a CVS file system which means that you don’t deal with files the way you normally do. To manage files you use a web browser interface (which is a darn good one I might add) but sometimes you actually want to edit files in your standard desktop application. And it is here the AL comes to play. You can download a file through the browser interface so that the file is opened in the AL. AL will then open the file in the desktop application you have associated with the file’s content-type. When you make your changes and saves them the AL will directly upload the changes to the server. So in short I could have said: The Application Launcher is a means to edit files on a remote Roxen server with a preferred desktop application.

The obstacles

I must say I’ve learned a lot from this project!

First off: If you download a file for editing and the AL is already started you don’t want to start a new instance of AL (this is something I have never ever thought about before – in general terms, not just concerning AL) but when you do think about it you find that it’s not a piece of cake to solve. I solved it the way it is solved in the original AL. The first instance of AL that is started also starts a “socket server” that listens for incoming traffic on a given port on the local IP. When a new instance is started it first checks if it can connect to said port and if it can it sends the arguments through the socket to the first instance which then handles the request. The second instance is simply terminated when it has send the data though the socket.

So there I had to do some socket programming. Great fun :)

Secondly: Stuff happens in the background of AL – data send through the socket remember – which means that nothing happens when you try to update the Graphical User Interface. (NOTE! This is the first more advanced desktop application I’ve done.) After “Google-ing” around a bit I came to know that this was a real newbie problem ;) The thing is that the GUI can only be updated through the same thread that started it so when using background threads – implicitly that’s what I’m doing although handled by the asynchronous callback infrastructure of C# – you need to make sure the GUI is updated through the main thread. This is the most simple way so solve it:

3 lines of C#
  1. Gtk.Application.Invoke(delegate{
  2. CallFunctionToUpdateGUI();
  3. });

That’s not too difficult when you know it ;)

Thirdly: The AL is sending data back and forth through the HTTP protocol which means we have to use some sort of HTTP client. C# has a couple of ways to do this but unfortunately they came up short, or I couldn’t use them anyway. I didn’t manage to figure out exactly why I always caught an exception saying something like: A protocol violation occurred!. I’m far from the only one who have fought with this and it has something to do with the headers sent from the remote server. You can invoke “unsafe header parsing” but that was to much of a hassle so I created my own little HTTP client.

One big annoying thing with C# is that is seems almost impossible to turn data from streams into strings without having to use any one of the System.Text.Encoding.* classes/objects which in my case meant that images and files in binary form got seriously fucked up. I manged to solve this my never turning the data into a string but keeping it as a System.Text.Encoding.* all the way from request to response to saving to disk. It was rather irritating but at the same time nice when solved (and I learned a whole bunch about System.Text.Encoding.*, System.Text.Encoding.*, System.Text.Encoding.* and System.Text.Encoding.*.)

Finally: Of course I learned a great deal more about C# but this blog post is starting to get pretty excessive so I will round it off by saying that MonoDevelop is starting to become pretty darn good! I just upgraded to the latest version of Ubuntu and that also meant that I got the latest MonoDevelop and I must say it’s more stable than ever (although it occasionally crashes) and a whole bunch of new features are in place. One I havn’t used before – although it might have existed before – is the “Deployment” stuff. It creates a package with configure and make files for optimal compilation. Really smooth!

Source and screens

I will finish off by adding the source files and a few screen shots:

Roxen Application Launcher 17:31, Sat 17 October 2009 :: 96.7 kB

Screen shot 1: Just a standard view
Roxen Application Launcher 1

Screen shot 2: The panel applet in action
Roxen Application Launcher 2

Screen shot 3: The Application Launcher in Swedish
Roxen Application Launcher 3

Screen shot 4: Adding support for a new content type
Roxen Application Launcher 4

PLib, the Poppa PHP Library

Other than working my ass off at work I just lately took a real approach on my secretly sleeping project PLib. PLib is my own PHP framework with functions and classes to ease my everyday PHP programming.

To give you an idea of what PLib is all about let me give you an example:

18 lines of PHP
  1. <?php
  2. require_once ‘/path/to/PLib.php’;
  3. //! Lets create thumbnails of all JPEG images in a directory
  4. PLib::Import(‘Graphics.Image’);
  5. //! The Dir class is included in Image so I don’t explicitly need to
  6. //! include it
  7. $dir = new Dir(‘/path/to/jpegs/’);
  8. $mw = 150; // Max width of thumb
  9. $mh = 90; // Max height of thumb
  10. while ($file = $dir>Emit(‘*.jpg’)) {
  11. $img = new Image($file>path);
  12. $img = $img>Copy($file>path.‘/thumb-‘.$file>name)>Scale($mw, $mh);
  13. wbr($img>Path()); // The full path to the thumbnail
  14. }
  15. ?>

Rather easy I think!

AutoDoc

Another cool feature (if I may say so) is the auto documentation. If you quickly want a list of all available classes and functions just type PLib::Info() and you will get the list. Each class and function is then clickable and will lead you to the documentation for which ever class or function you clicked.

If you want the documentation for a specific class just type PLib::Info() and there you have the documentation.

The third way to use the autodoc is to pass an instance of a class as argument like PLib::Info().

And if you wan’t a full documentation site just type PLib::Info() and you will get the same documentation as available at the PLib site.

The site

Anyway, PLib has its site of it’s own where anyone curious can download the latest version of PLib and find out more about the project.

plib.poppa.se

Script for adding all new files in a SVN repository

When working with revision controlled stuff you sometimes add loads of new files to an already existing repository. Then when you are about to check in the next revision you have forgotten where all the new files are and you probably don’t feel like writing svn add thefile too many times. This is of course no problem if you’r using a decent SVN client – like Tortoise SVN on Windows – but I prefer the command line so the solution was writing a script that adds all new files in a directory structure to the SVN repository.

There are “one liners” for this but I wanted a bit more flexibility and ease of use so I wrote the a script that also lets you define regexp patterns for files you don’t want to add even if they exist in the directory structure.

This is how to use it:

7 lines of Bash
  1. # Add all new files
  2. svnadd path/to/repository/
  3. # Add all new files except those matching the regexp
  4. svnadd -s “.*.txt|tmp/.*” path/to/repository/
  5. # Add all new files except those matching the regexp in
  6. # the file defined by the “-f” flag
  7. svnadd -f regexp.txt path/to/repository/

Quite easy! If you don’t feel like remembering the pattern for files to skip, put the regexp in a file and use the svn add thefile flag.

77 lines of Bash
  1. #!/bin/bash
  2. ##########################################################################
  3. #
  4. # Script for adding all new files in a directory structure to an SVN
  5. # repository.
  6. #
  7. # copyright © 2007 Pontus Östlund <spam@poppa.se>
  8. #
  9. # The svnadd script is free software; you can redistribute it and/or
  10. # modify it under the terms of the GNU General Public License as published by
  11. # the Free Software Foundation; either version 2 of the License, or (at your
  12. # option) any later version.
  13. #
  14. # The svnadd script is distributed in the hope that it will be useful,
  15. # but WITHOUT ANY WARRANTY; without even the implied warranty of
  16. # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU General
  17. # Public License for more details.
  18. #
  19. # You should have received a copy of the GNU General Public License
  20. # along with this program; if not, write to the Free Software Foundation,
  21. # Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301, USA.
  22. #
  23. ##########################################################################
  24. while getopts “s:f:h” opts
  25. do
  26. case $opts in
  27. s) skip=“$OPTARG”;;
  28. f) if [ -f “$OPTARG” ]; then
  29. skip=$(head -n 1 $OPTARG)
  30. else
  31. echo “$OPTARG doesn’t exist!”
  32. fi
  33. ;;
  34. h) help=1;;
  35. esac
  36. done
  37. shift $(($OPTIND1))
  38. if [ -z $1 ] || [ “$help” ]; then
  39. echo “About: svnadd enables you to add all new files in [dir] to a “
  40. “subversion repository”
  41. echo “Usage: svnadd [flags] [dir]”
  42. echo
  43. echo ” -s <pattern> Regexp pattern for files to skip”
  44. echo ” -f <path> Path to file containing regexp for files to skip”
  45. echo ” The regexp should be placed in line one!”
  46. echo ” [dir] The path to the repository”
  47. echo
  48. echo “Example: svnadd svnsrc/”
  49. echo “Example: svnadd -s ‘tmp/.*.xml’ svnsrc/”
  50. echo “Example: svnadd -f regexp.txt svnsrc/”
  51. echo
  52. exit 0
  53. fi
  54. dir=$1
  55. if [ ! -d “$dir” ]; then
  56. echo “$dir is not a directory”
  57. exit 1
  58. fi
  59. while read status file; do
  60. if [ “$status” = “?” ]; then
  61. res=$( echo “$file” | egrep -o “$skip” )
  62. if [ -z “$res” ]; then
  63. svn add “$file”
  64. fi
  65. fi
  66. done<<EOF
  67. $( svn st “$dir” )
  68. EOF
  69. exit 0

Download

svnadd 17:31, Sat 17 October 2009 :: 2.1 kB

PHP documentation browser

Even though I have my vacation right now I need to hack some ;)
What I have done is a PHP documentation browser, I simply call it PHPDoc Browser, for GNU/Linux to read the PHP documentation locally on the computer (it’s a little bit like CHM for Windows). This little app isn’t that necessary but I thought it was a great thing for learning new stuff: The application is written in Mono/C# and uses SQLite for storing the search index.

Difference from CHM

Although there’s a CHM equivalent, XCHM, for Unix like systems I decided that my application serves a purpose: What I like about the PHP manual is that you can hit www.php.net/the_function in you browser and you will get the documentation for “the_function”. Of course I implemented the same functionality in PHPDoc Browser. You can also do a wild card search like “array_*” and you will get a list of all “array_*” functions.

I also implemented a free text search but that’s not “a real” FTS at the moment. FTS in SQLite in a little bit harder than in MySQL so while awaiting the next SQLite version the FTS is a bit of a hack. But you can quote the search to narrow it down ;)

What’s learned?

Most of the unnecessary stuff I do I do to learn and the PHPDoc Browser is no exception. I learned how to use SQLite in Mono/C# and I also wrote a self contained installer in Bash, and Bash I havn’t really touch although I’ve been using GNU/Linux for 6-7 years! I found it quite fun writing the installer :)

Howto

If you would like to try it out:

  1. Download the install script
  2. Un-tar it (`tar zxvf phpdocbrowser-installer.tgz`)
  3. Make sure the script is executable (`chmod +x phpdocbrowser-installer`)
  4. Open a console and run the installer: ./phpdocbrowser-installer
  5. Answer the questions and you’r ready to go.

The script will create a directory, ./phpdocbrowser-installer, in your home directory in wich you will find ./phpdocbrowser-installer, ./phpdocbrowser-installer and a directory, ./phpdocbrowser-installer, with the PHP documentation. A “run script” will also be installed either in ./phpdocbrowser-installer or in your home directory depending on how you answered the questions and if you agreed to create a desktop shortcut one will hopefully appear on your desktop (havn’t tested this on KDE but it should work fine in Gnome).

To run the application hit the desktop shortcut, if one was created, or invoke ./phpdocbrowser-installer from a console.

If you wish to remove the PHPDoc Browser run the following from a console:
./phpdocbrowser-installer.

And that’s that!

Download

PHPDoc Browser 17:31, Sat 17 October 2009 :: 8.9 MB

Screen shots

PHPDoc

PHPDoc2

PHPDoc3

Bug fix for Syntaxer.pmod

It didn’t take too long to notice that I had f–ked up the HTMLParser class a little bit. It was how entities was handled that didn’t really worked as expected – entities in tag attributes was duplicated and inserted in the tag content – but the good side of it is that I learned about the HTMLParser method of the HTMLParser Pike class. I only want to match entities in the data section – i.e. tag content – and not in attributes and the HTMLParser method tells you, as the name implies, in what context the entity is found. So my entity callback function now looks like:

6 lines of Pike
  1. //! Entity callback
  2. protected void ecb(Parser.HTML p, string _data)
  3. {
  4. if (p>context() == “data”)
  5. line += colorize(entify(_data), “entity”);
  6. }

which hopefully will be completely bug free now. So one down 854 to go ;)

Codify RXML tag and Syntaxer.pmod 17:31, Sat 17 October 2009 :: 183.9 kB