It took me whole 6 hours today to work out this totally new way of generating PDF documents in PHP. Earlier I had been using dompdf – which is very good except it has limitation around complex HTML and TABLE tags. Since I usually get a PDF template in a non HTML format (psd or jpgs or pdfs), its always a struggle to make a good enough HTML for use with the converter. I’m on ubuntu and therefore needed a free solution.

The tricks are as follows:

1. Convert the PDF template into SVG format.
   I use inkscape in ubuntu for the conversion. If a direct conversion is not possible, such as DOC files then convert DOC to PDF (using OpenOffice maybe) and then use Inkscape to load PDF and save as SVG. Do this step anyway you can using any software you can find.
   The SVG will very nicely preserve the same exact looks but now it becomes editable. Its neat XML – that is what we want.
2. Identify your placeholders.
   A tempate usually will have text place holders within. (Placeholdes are those pieces of text which change in every generated PDF.) Use inkscape to mark labels to these pieces of text. Labelling is needed so you can XPath the node directly and change the nodeValue. Save the SVG as a new name (for backups). This new file we take forward. If you have a dynamic text list where items count change with every PDF, it gets little harder. In my case I created 5 rows in SVG itself. They are converted empty in some cases and that alright with me. Otherwise, youll have to study little SVG.
3. Replacing placeholders with real data.
   IMPORTANT: Before XPath can use the SVG, you need to remove xmlns attribute from the svg element. Do this in a normal text editor. Without removing xmlns this XPath queries won’t fetch tags with labels because they are with a namespace (inkscape:label). This editing will be needed everytime you modify your master SVG from within inkscape. I spend 2 hours to understand why my XPath query was not returning data in my SVG. If any of you know how to fix this xmlns (default Namespace) problem with XPath, please suggest.
   Within PHP, load the SVG using DOMDocument object. Then create a new XPath object using that. Create XPath-queries for each place holder in your SVG file and replace DOMElement-nodeValues with your real data. Now save your DOMDocument as a new SVG file. You don’t want to replace your master SVG with new data.
4. Inkscape conversion.
   I called inkscape using php system command “inkscape my.svg –export-pdf=my.pdf”. It generated my PDF. There can be many command options you could choose for inkscape but thats beyond this blog.

The problem I had was editing the SVG text nodes nicely to support dynamic text. Earlier generated PDFs had text too conjested or over-written. Also, SVG does not support wordwrap. You will need to create lines and then fill lines yourself. hint: php has a wordwrap function that I used for word wrapping long lines. Inkscape has windows binary too for non *nix users. The trick is language independent. DOM support just helps.

Thats all there is. I did not share/write code in this blog (tired). Ask me and ill share some code/sample too. This trick definately worked very nicely for me.Step 3 above could also be done without using XPath. You could direct call getElementById. I felt hard coding IDs of SVG in my PHP code could cause problem if I were frequently editing my master SVG. There are even other ways to change SVG.

There was once when we needed a LAN chat server. After hunting located Openfire which is an XMPP implementation. I did have the knowledge of it being an Open stardard but thats all i knew then. That was more then a year back. Now we have a dozen servers across the city and all running OSS software on a medium grade desktop machine hardware. We use the same server to chat with all the staff and archive messages.

After installing the second server, I knew, it needed a self health reporting system so we sitting in the moon can see if a problem is developing or even the reason for a problem. There are all sorts of problems that come on servers that are not in a protected space and without a stable internet or network connections. The first idea was simply to make a webpage and dump vital stats. But who the hell will go to a dozen IPs for checking if it is fine. Thats where I knew, my Openfire XMPP was the key for a fantastic new way of realtime monitoring and much more.

Just like any chat servers, it also supports making rooms. These rooms can be joined by anybody and can simply see the activity going within. Thats the key. I created a health-check room. I developed by entire stats gathering core and piped it using XMPPHP library into the targetted room. Then scheduled a script to run every 5 minute. Now the result is that my room can be joined by any person who wants to monitor the current stats. The stats gathering core has been scheduled in every installation with a gap of 30 seconds. Therefore, all servers send their reports in a gap of 30 seconds ina  continous stream within the room.

The above practise was used perfectly till I realized something more. Openfire has inbuilt support to archive messages. Therefore the stats was not parsed with details instead left as is within its Mysql DB. The same DB later was later also the point where remote comman output stream was sent to. Today we have 3 rooms for realtime stats, health stats and one exclusive realtime room for test cases, runtime errors in PHP/Mysql server. This is far better than RSS fees since it refreshes itself automatically in a stream and sourced from the servers itself. Far easier to use. An RSS feed was also made just for my mobile which parsed data from XMPP archive DB.

My favorite XMPP client is PSI. Its sleek, java-less (which means loads faster with less RAM) and with better UI. XMPPHP is a great library to access and conduct all operations using the standard. It happens that its docs don’t have everything it is capable of. I can share some of my code on request, specially the groupchat code which was not easy to locate.

The problem comes for various reasons. The one that attacted me was since I never added the subdirectories of the /app/tmp directory into my subversion repository. The name tmp suggested me that it would be recreate the entire required directory  structure. But I guessed it wrong. After hunting the solution and not able to find it, I simply realized that the default install of cake has content below the tmp. and, sooner I placed it back, cakephp home page was happy. The directories structure is as:

• app
  • tmp
    • cache
    • logs
    • sessions
    • tests

Simply make sure tmp directory structure remains the same. As suggested by the cakephp error, core.php won’t actually help in this particular case.

MVC model is what I never tried my hands on. The ModileTrailExplorer is also based on the same framework but I did not work on the core. Ruby on Rails in entire on MVC and then again I did not work on that either. Me a PHP guy who creates his class libraries for the UI of the code. So, how can I experiment on MVC was the long standing question. Finally learned about CakePHP from many guys on OSSCamp and other places. And there start the story.

The installation is easy. But based on the configuration that I have understood about cakephp, I had to give it a seperate VirtualHost pon my apache. The reason also being that my trunk root is already using Joomla. Joomla means it comes with its own mod_rewrite rules. CakePHP also has its own set of rules. Therefore Im still confused of mixing the two. Currently the application Im working runs on a single domain. And therefore, Ill have to understand to configure both Joomla and CakePHP togerher on the same DocumtnRoot.

Then comes the fact to actually make a small form which saves datta to my table. CakePHP runs on its own database naming convention. That is a hurdle since my application is old and already has its own unique convention. So I had to learn about AppModel configurations which will allow me to give custom table names, primaryKeys and even foreignKeys. Actually, till now (my fourth day of cooking) I have spend least 60%  of my time configuring AppModels. Feels like to get cakephp on an existing application as well as on an existing database means loads of homework in AppModels. I believe once my required Models are done, my application coding can take its own pace. But for now, its very slow. api.cakephp.org is my reference storehouse for now. For a newbie, every 5 mins I have to hunt how-to-do-the-foo-bar thing in cakephp. This is not a point of discouraging, you need to learn as well as port. Both things as the same time will be very very slow. All depends on your experiance level and speed to learn.

Another point to share is that I had my exclusive class library in my generic PHP application. That class cannot and will not be ported to cakephp cause its become the most complicated engine. Importing an existing class library and include files was done using the the bootstrap.php. The config part is still not done in a neat way but somehow its running without making me to copy the entire class library into cake. I would prefer it that way so that we don’t start maintaining two codes of the same files.

more updates as time goes. the project is doing good and i like many things already. will list them soon.

Really! Thats what I just noticed. The test machine #1 is my XP has no firewall, is a 1Gpbs intel ethernet card on a 100mbps network. The test machine #2 is a ubuntu 7.04 workstation again on the same network as me with its own static IPv4 address. The great server is also ubuntu 7.04 with mysql 5.x. Now the case noticed was that telnetting from my xp (#1) to server at port 3306 took least 6 seconds for the handshake response from the mysql server. Telnetting from the ubuntu (#2) to the server at the same port actually responded the moment I pressed the return key. Ethereal’ed the whole transaction to find what strange in #1 TCP packet and the #2 TCP packet. Did not notice anything great. Googling on this also did not result any problem of this sort. But the results are true. Ubuntu connection to 3306 is taking under 5ms compared to XP which is taking least 6000ms. There are no slow down process running except maybe TeaTimer.So makes me wonder. Im sure no guru in TCP so i can’t dig more on this.

The reason why we had to conduct this test in my office is cause of a typical development floor setup. There is just one Mysql on the network and clients use their localhost apache’s to connect to this remote server. Server ping takes around 0.3ms with no loss. The guy on XP was killing himself due to the delay it took for his localhost to render php database pages. The other guy on ubuntu was on mercedes. His apache ran like he owned everything on his machine itself. So there it is. Since google did not report any similar case with anybody else, I await if someone can share this case. Cya!

It was a small handbook I got hold of at Ramakrishna Mission Institute of Culture, Gol park, Calcutta around 1998. The book was Visual Basic 4 Tips and Tricks or something like that. Then I had just started my NIIT’s GNIIT course. The first semester had started which was about DOS and all other crap. I had good knowledge of QBasic syntax before this. Anyway I had done lot of DBase III Plus programming and even played with C using the K&R – The C Programming language. Therefore, the syntax and fun that I learned from book got me excited to play with real VB4. That was the first time I had made windows program and a MsgBox “Hello world”. Wow. I had learned this too easy language syntax for RAD within the next 3 months. We had a picture box. A grid based windows form editor. Later in after an year I had to enter Visual C++ SDK programming followed by the official study of RAD using VB4. ha.

Here is another account of a very irritating experiance I had with the c# language. A guy from C really has to struggle with stupid non-geek compile errors. check this code. This was very correctly written for C. All i wanted to do was save my object code with duplicate code.

…

if (Request.QueryString["CID"] != null)
CategoryID = Int32.Parse(Request.QueryString["CID"].ToString());
else if(artwork_id>0)
{
CategoryID = Artwork.GetArtworkCategoryID(artwork_id);
    goto tr;
}
else
{
CategoryID = Artwork.GetCategoryID(ArtistID);
tr:
Server.Transfer(”work.aspx?AID=” + Request.QueryString["AID"] +
“&CID=” +CategoryID + (debug?”&debug=true”:”"));
}

…

all that compiler said was  error CS0159: No such label ‘tr’ within the scope of the goto statement. I do understand that jumping to other code is to be done carefully.  you might skip some variable initialization or derive into some invalid stack. Having siad that, an average coder would know all that. Therefore, jumping to any code using goto should be very much allowed by default.

Setup is easy. Graphics are neat. Documentation is good but with not beat for first timers like me. I have had experiance only with EPiServer before this one. That is a windows dot net CMS. Its fair but besised being closed source, it ignores the performance aspects. TYPO3 is open so we can’t complain. (I like this feling). Has template support and also a nice typoscript language too. The language is very simple just like advanced configuration script. There are loads of extensions too.

It took 3 full days by the time I could run my first few page with one most simple menu and a simple template. That was fine till that point. Beyond that, I need my own custom menu and very complicated templates. Documentation for doing these is very less. I have to figure out the trick based on what what expansion API are supported. Tutorials are good but none for thinking differently.

Today is day 4 and I have got to finish the menu hacking exersize. I need to compose my own menu with my own choice of wrapping tags and content as already built images.

This trick was developed in one of my biggest database. It has 7 million records with 160 fields each. Half as text fields. the size of database after converting to best field size and database had come from 17GB (raw fixed width text) to 5GB MYI with neat RDBMS. Data was 35% while indexes were 65%. It was like this for best search performance. Now the text data like First Name and Last Name were problems. They take ~25 letters each and indexing them was not solving most of the problem.

I knew MySQL like operator was case insentivite. This may sound like good news but it’s not. For every search it anyway has to setcase the field of every record before applying the search function. MySQL is very good in group indexes. Suppose you are searching for vehicle type 17 and widht as 19, a index which uses both will have better performance than index of both independently. Secondly, fixed width indexes are too fast then character. Numeric comparisions are even then varchar comparisions. My problem was to search First Name matches.

The trick uses all the ticks together.

1. I made fiurst three letters to upper case.
   
2. I converted first three letters into a 16bit integer (field F).
   
3. Created field L as the balance letters from first name.
   
4. Created index A with field F as the first and field L as the second.
   

VOLA! Mysql loves me. Explain listed that to scan a particular first name, it did not need a table scan any more. First three letters used to make the scan into very small number of records. Also first three letter were index therefore easily approached from the index.

There must be a question of how was three upper case letters converted to 16 bit integer. Well, it was 15 bits that were used. 26 possible letters means 5 bit scope. Multriply that with 3 letters and we have 15 bits.

I’m badly looking for a method so I can extend or create my own firmware for the camera of my choice. All camera have their own RTOS and hardware. Therefore I should not expect same SDK for many camera companies. But I can expect same SDK for many models since same companies very much use same environment. More critical than anything is that the manufacturers hate to offer any such feature. I have a Casio EXILIM EX-P505. The camera is very good. I can’t say that more. Its so good that I want to extend it. I use it for Time lapse imaging. If I could automate the clicks for every 60 seconds or so, it would be very easy to do such photography. Yes, web cams can help me on that but web cams are very low of resolutions. Added they will need a whole computer along therefore no portable at all. I have been googling a lot for my cam model and few others but not found any crack to the problem. Some have parsed the firware update and split the files into ROM and JPG and music etc. ROM is assembly compiled written most likely in C. Reading the ROM of my casio i find paths as A:. Means the RTOS is very much like DOS STDLIB. That is good news but again getting a compiler or decrypting the compiled code is going to be hard. Then there are other problems as missign documents and interrupts. What button does what. How to trap events. All will have to be reverse engineered. Pain Big Time. Its sure possible for Pro. And thats not me! So I wrote this blog so anybody of similar interests or with leads on thsi topic can have some thoughts. I will write more on this topic soon.

Blogged with Flock