Tuesday 30 January 2007

The Perfect Nightmare - John Saul

This is not one of John Saul's best books, but then again, when you say John Saul you say cliche book. All his books contain some sort of drama, usually concerning children, always in a backwater town or suburb, always playing the child in danger, family connection, deep instinctual fears, etc. But some of them are more interesting than others, me liking science related books more, even if I almost always relate to the mad scientists and their evil creations than to the pseudo-moral rednecks that fight against them. This one is plain boring.
The Perfect Nightmare is about a child abductor. He goes for young innocent girls, he takes them into a makeshift playhouse and "plays" with them. Being a mass production book, it presents some brutal violence, almost no sex at all, even if it hints towards it, and the first and largest part of the book is simply puffed with the actions and fears of the people left behind which are mostly average boring people. This makes the book average and boring. The murderer himself is not something truly original, neither is the way he is portrayed.
The ending of the book has a catch, but after reading 90% of it, it doesn't really has any effect. Bottom line: not even for John Saul fans.

Monday 29 January 2007

Saturday 27 January 2007

Event Bubbling in ASP.NET 2.0 and templates

I have been working recently with Page.LoadTemplate and ITemplate.InstantiateIn to add content from user controls to pages or other controls. This thing solves the problem of having to declare abstract classes in App_Code, but poses another problem linked to the lack of object references to the user controls you instantiate. What that means, among other things, is that you lose the ability to use events in your code.

But there is a way to bubble events to the upper controls and catch them. The first thing you should look at is the OnBubbleEvent protected method of the Control object. It catches all the events raised by its controls. Override it and you can catch events and treat them. Now, this method returns a bool value. It tells the RaiseBubbleEvent method (I'll talk about it shortly) if the event was handled or not. If it was, then the event doesn't rise above that control.

Now, about the RaiseBubbleEvent method, also a protected method of the Control object. What it does is go up the object hierarchy and execute on each object the OnBubbleEvent. It also stops if the method returns true. It is so simple and perfect that it is not overridden in any of the ASP.NET 2.0 WebControls.

But there is a catch. Some objects, like DataLists, DataGrids, Gridviews, Repeaters, etc, choose to handle the events in their items no matter what. That's why if you click a button inside a DataList, the event won't bubble to the OnBubbleEvent method from the UserControl or Page the DataList is in. Instead, the event reaches the OnItemCommand event.

So, if you want to use this general bubbling method with WebControls that override OnBubbleEvent, you have two options.
One is use inherited objects that remove this issue altogether (like inheriting from a DataList, overriding OnBubbleEvent, executing base.OnBubbleEvent, then always return false), which might make things cumbersome since you would have to define a control library.
The second option is to always handle the OnItemCommand and other similar events with something like RaiseBubbleEvent(sender, args); . This very line will re bubble the event upwards.

Benefits: the benefits of using this general event bubbling technique is that you don't have to always specifically write code in the OnItemCommand or having to handle OnClick events and so on and so on. Just catch all events, check if their arguments are CommandEventArgs, then handle them depending on source, command name and command argument.

Thursday 25 January 2007

The Twin Towers, national testicles!

I know it's bad taste, but it was too good not to blog about it. I just had this vision of this school bully, pushing kids around, hitting and scaring them. One day, one of the kids fights back and kicks the bully in the nads. So he crumbles to the floor, but then tells everyone how that attack was unmanly and cowardly and then beats the crap out of the kid.

The same thing happened with the 9/11 thing. Two every important American items, symbolically destroyed to show the otherwise almighty US that they can be hit, even by the little people. It hurt like hell and it made everybody angry, but it also made the point.

The vision of the twin towers, as giant testicles (with small sperm inside?) felt hilarious to me...

Tuesday 23 January 2007

Piata Unirii (Unity Square) in Bucuresti (Bucharest)

Agent Smith from Matrix once said "I hate this place. This zoo. This prison. This reality, whatever you want to call it, I can't stand it any longer. It's the smell, if there is such a thing. I feel saturated by it. I can taste your stink and every time I do, I fear that I've somehow been infected by it" and that made me like him instantly.

He described perfectly how I feel whenever I exit the subway or bus at Unirii and go towards the tramway 32 station. It's only 100 meters, maybe 150, but you have to fight your way through a river of people, a lot of them smoking, then wait for the green semaphore light a long time while smelling all kind of interesting flavours, then try to get in the tram without being rubbed away like a pencil trail by people who don't really care if they touch you or not.

If you are lucky enough and it's winter time, maybe before some holiday or another, you get to smell the homeless people who decide that either it's too cold outside and camp in the tram, either that you have to give them a holiday present and start begging and singing and other things which basically mean they also move or crawl, thus evenly smearing the air with their fetid stink.

Dance, Dance, Dance by Harumi Murakami

The lead characters of this author seem to always be quiet guys, ready to accept anything and be taken anywhere by the story. The same incredible things happen, all narrated in a very calm way, making them real. The obsession for American or English music permeates everything. I've read two Murakami books, but also some reviews for other books, and the same situation seems to be repeating itself.

Dance, Dance, Dance is about this kind of a person, caught in a kind of metaphysical quest to find himself. Some symbolism, weird characters, absurd situations. In the end, surprise, he finds himself. Compared with Kafka on the Shore it is much slower at the beginning, but it picks up pace closer to the end, yet the very end is slightly different from the rest of the book. I first though that Kafka was a better book than this one, then I switched sides and now I see them as of similar quality. Not much else to say.

Now I can't say I didn't like the books. They are the kind of stories that take you nowhere, but on a very pleasant road. But in the end, you are left with nothing. Nothing really learned, nothing really gained, just like a walk in the park. And I rarely enjoy walks in the park.

So I have decided to call it quits. Harumi Murakami is a good writer, there is no doubt about it, just not my type of a writer. I feel his writing is too English and not Japanese enough, thus defeating the very purpose I started to read his books in the first place.

Monday 22 January 2007

Use different templates in the same DataList

First of all there is no reason not to apply this method to any other templatable control like GridViews or DataGrids. The basic mechanism is the same.

The problem here is how to display different templates for different items (not just for different item types) like when databinding to a general collection of different objects.

Basically all you have to do is use the OnItemCreated event to first load a template (using Page.LoadTemplate) from any user control and then applying it to the DataListItem (using ITemplate.InstantiateIn). But there are a few catches. First of all, the original templates (the ones in the as?x file) are applied before the OnItemCreated event. So if some templates are defined, like ItemTemplate for example, you need to clear the controls in the item before you instantiate your template. Second of all, you cannot declare the OnItemCreated event in the Page_Load method, since it is executed after the OnItemCreated event. But you can declare it in the as?x file.

So let's combine all this into code. First the aspx:
<asp:DataList id="dlMain" runat="server" OnItemCreated="dlMain_ItemCreated" >
</asp:DataList>

nothing fancy. Then the cs code:
protected void dlMain_ItemDataBound(object sender, DataListItemEventArgs e)
{
ITemplate template1 = e.Item.ItemIndex % 2 == 0
? Page.LoadTemplate("~/ucTest1.ascx")
: Page.LoadTemplate("~/ucTest2.ascx");
e.Item.Controls.Clear();
template1.InstantiateIn(e.Item);
}


That's it! ucTest1 and ucTest2 are two arbitrary user controls I am loading alternatively. For this particular case an AlternatingItemTemplate could have been used, but you get the point.

Sunday 21 January 2007

Editing HTML directly in your browser!

Check out this interesting piece of code:

javascript:document.body.contentEditable='true'; document.designMode='on'; void(0);

Pasted into the address bar of browsers like IE6, IE7, FF2 (haven't tested on others, but I think it should work on most) it allows the editing of HTML directly on the page. You can expand and shrink HTML elements, you can edit text, etc.

While on Internet Explorer the changes are not saved if you save the page, on FireFox you get spell checking while editing (with red underlines for unrecognized words) and you can save the changes with save page.

It's really weird, check it out.

Saturday 20 January 2007

Experimenting on humans

I just finished watching a very cool documentary, called The Human Behavior Experiments, that I highly recommend seeing. It describes some of the mechanisms of the human mind that make us behave inhumainely, or iresponsibly and the experiments to expose them. I found very interesting the information on Wikipedia about these experiments and I will post a list of links below.
Doing the research, I've stumbled upon sites that describe (or plainly show the pictures) of the Abu Ghraib tortures. Those links I will not publish, but you can easily use Google to find them yourselves. It is incredible that after those things got published, people still support any war or detention center at all. The movie I was telling you about explains how things like blind obedience or difussion of responsibility function in most of us, ordinary people.

Links:
The Human Behavior Experiments
The Milgram Experiment
The Stanford Prison Experiment
Das Experiment
Abu Ghraib abuses
The Bystander Effect
Asch conformity experiments
Depersonalisation
Responsibility diffusion

Thursday 18 January 2007

Damn telemarketers!

Telemarketing is already embedded in the western popular culture, but in Romania, we didn't have it until recently, and only very rarely. Usually they didn't want to sell stuff, but to poll opinion. So I was immediately annoyed when a authoritative male voice started to talk to me about increasing the money I get from my insurance. That annoyance increased when the voice continued "if you want to... bla bla bla, press 1". They didn't even bother to put a human to give the phone calls, it was all automated. Swearing at a machine would have been ridiculous, so they took away the only possible emotional outlet. They have denied my anger!

So of course I resorted to THE BLOG [lots of echo here] to shout my frustration! But since I have worked with telephony in my days, I thought "wait a minute, there is a solution". I could strike back. First of all, I would install an answering system that requests the user to press a key. "If you are a telemarketer, fuck off, if you are a human being, press 9". Then add the tone of the key "1" and repeat the message. But then again, why would I bother, for one lousy telemarketer? Because they took away my outlet! I want it back, damn it!

Bookmark AJAX pages.

One of the issues that largely remains unsolved in AJAX applications is the ability to bookmark (say, add to the Favourites) the page with all the AJAX added info on it. This very cool javascript library allows one to save the state of the page using the window.location.hash variable, which holds the string after the # character, also called a hash.
Javascript can modify and read this value with no refresh of the page. Also, if you have anchors in your page, like... <a name="hash" Javascript code like
window.location.hash='hash';
will scroll the page there.

Tuesday 16 January 2007

Using Web User Controls within a web service or class.

I have this Web User Control that has a simple function of displaying a customised grid and fill it with data. Since I had to email the rendered HTML I've added a static method to the Web User Control that would render itself based on some parameters. However, in Net 2.0 one cannot dynamically use the types declared in the codebehind of a user control or page inside another control or page, only types declared in App_Code. In pages this gets solved as in the link above, but what do you do when you want to use the type inside a class in App_Code or, like I needed to, in a web service (where register directives are not allowed)?

In my case, there were two solutions. One is to create a page that loads the user control and nothing else, then read its html. That solves the problems of using the code, but adds security issues. I have to either add a security mechanism to the page or allow anyone to render the UserControl. The second solution, the one that I ended up implementing, is to use reflection.

Let's recap. I have the web user control, let's call it Wuc, and I have the static method Wuc.GetHtml(int i) that I must access from a Web Service. If I write
string s=Wuc.GetHtml(i); it doesn't get to compile, returning the error "The name 'Wuc' does not exist in the current context".

So I do something like this:
UserControl uc = new UserControl();
uc = (UserControl) uc.LoadControl("~/Wuc.ascx");

and I get the object I need to get the type of. Then I should get a reference to the method I want so I try:

MethodInfo mi = uc.GetType()
.GetMethod("GetHtml",BindingFlags.Static);


which should work, but it doesn't!

Why not? Because the type of the object is not Wuc is wuc_ascx and it's the dynamically generated ASP.NET type. I get all the methods of Wuc, but not the static ones! So, the (really ugly) solution is to make the method not static and use this code:

MethodInfo mi = uc.GetType()
.GetMethod("GetHtml");
string s = (string)mi.Invoke(uc, new object[] { i });


which finally works.

Update
Scott Gu was nice enough to respond to my request on how to do this. He has this blog entry that explains how to render a control in NET 2.0 within an ASP.NET Ajax environment, but what really connects with my blog entry is this archive and the ViewManager object. Scott uses here the Page.LoadControl method (here is why) to load the control and Server.Execute instead of RenderControl.

Monday 15 January 2007

The dogs in Bucharest

I've just remembered that I had to write a long overdue blog entry about the dogs of Bucharest. Bucharest is an old European city, meaning that it grew with the needs of the people, starting from a little village and becoming Romania's capital. A lot of things happened here, like the world wars. During the war a lot of pets were left homeless and ownerless and became street mutts. That created in time a large population of dogs, living in packs, having their own territory and so on and so on.

Now, fast forward to the Basescu era. This guy was so frustrated that Bucharest was not looking like most western European capitals, that he focused on transforming Bucharest, starting with what he probably saw while he was abroad. That meant getting rid of dogs and boutiques and street merchants, amongst other things. Well, I went for a small tour of Europe a while ago, and I didn't like it. All the cities were barren, like lifeless stone deserts, where you could only pay a lot of money for the smallest amount of food or walk aimlessly looking at old buildings. Basescu wanted to turn Bucharest into this.

A lot of people were for this getting rid of dogs business, while in the mean time, animal lovers were crying against it. In order to truly make a decision, one must know what the dogs of Bucharest are like. Most of them are very friendly, they let people pet them, while begging for food or attention. They are not wild animals, but liberated pets, and they act more like pets that way. Some of them, organised in packs, tend to become very territorial, defending their turf with loud barks and sometimes (when the human doesn't get it) with bites. Because of this kind of behaviour, most Bucharest dwellers agreed with the dog removal. But most dogs were having a very adapted life to the city. They had families, ways to get food that included being nice to humans as well as hunting and eating rats and, don't laugh, a culture of their own. I do say that because I have watched them behave and along with their natural instinctive behaviour and the stuff they learned during their life, they also exhibited learning and teaching abilities. I have seen mother dogs showing their pups how to grovel or how to cross the street and avoid cars.

But what does this say about us? That we want to destroy what we fear and don't understand. Even the more aggressive dog packs would not attack a human that doesn't provoke them and that faces them. A simple trick like walking backwards (and not trying to show the human superiority by kicking dogs or by throwing stones at them) would have left even a 20 dog strong pack barking, but not biting. And this kind of aggressive dog behaviour can be immediately fixed in each particular case. No, instead we chose to "nuke" the dogs, making us worse than the most bite happy mutt pack out there.

No dogs means also no life on the streets, no puppies except the ones in pet shops (the expensive ones that you have to buy because they have a pedigree and that get sick really easy), more rats. In some of the western countries that people like Basescu so blindly admire, dogs are being used in anger management programs in prisons. Taking care of an animal, just looking at a friendly, uncomplicated creature that craves your attention, gives us something that fights off anger and rage. We have enough rage in Bucharest, alright, but the dogs may soon be gone.

NASA goes metric! The ISS remains in limbo.

It appears NASA has used the metric system extensively since 1990, but now they've decided to not use anything else. NASA finally goes metric, so what a good oportunity to rant this is.

How about the funding? In 2002 the entire NASA funding was less than 15 billion, in 2006 it reached 16.5 billion. The total cost of the war in Iraq is 350 billion and climbing, with an estimated 600 billion in 2010. The International Space Station seems lost in bureaucracy limbo, and this site suggests the highly inflated figure of its cost reaches 100 billion. So what does that mean? That we could have skipped the Iraq war altogether and build six space stations instead? I am sure this is not the case. Because actually building six stations instead of one would have used only once the cost of design, aproximately the same people and each part would have been built cheaper than the ones before. Critics say that the cost of the ISS goes well above the first cost estimations and compare it to MIR, which has cost only 4.3billion. Well, that may be true, but doesn't that mean that we could have skipped the war in Iraq altogether (well, actually not all of it, Kuweit still had to be liberated) and build over 100 MIR stations?! Or finance 30 years of NASA budgets!

Well, had to be done. I feel much better now. Thank you, I will pay for the damages.

Sunday 14 January 2007

Sonic pollution and ear plugs...

A couple of weeks ago I bought myself some ear plugs, the kind that you stick into your ears, they expand like a sponge, and filter all but the loudest or low frequency sounds. At the time I was half ashamed of my reasons for buying it; first of all I planned to use them in bed, when I wanted to read and my wife wanted to browse the TV channels; second of all I wanted to filter out chit-chatting women and teensters from the tramway or buses. Yeah, I know, I sound like a grumpy old man, and that was the reason I was half ashamed of doing this.

However, now, after testing this new gadget and finding new applications every day, I am happy I made the purchase. I can read on my way to work in total silence, even with people talking and laughing as hard as they can in order for others to notice them, or people listening to music so loud on their headphones that you can't escape the sound even if moving further away. These babies even filter out the manele ring tones, spat out loudly by the latest model of cell phone.

Today I was especially glad I had my plugs on, since I boarded a bus that had the radio on. Pop music, silly monotone dance rhythms, loud and useless commercials that are even more spammy than spam email for the single reason you can't close your ears... they all went away. I now have that smug feeling of superiority about my ear plugs as I have about TV after getting Internet broadband and only downloading what I want and refusing to gulp all the idiotic programming the popular stations feed us.

Ear plugs rock!

Coins, applied to neural networks

I have always been fascinated by the concepts of Artificial Intelligence. I truly think that computerised brains are the next step of evolution, as biology (hindered by outdated morals, also) will never develop or evolve quickly enough. But enough of that.

I was thinking the other day of artificial neural networks, the electronic analogy to a brain, and the fact that for N neurons, you get to have N*M connections, where M is the average number of connections a neuron makes, which are in fact the important parts.
Click for details


Now, this is only an idea, Haven't thought it through, but if history is anything to go by, I will quickly forget it, so I'll just write it up. What if we could use a discreet interval for describing neural connections? Kind of like when paying money. You make thousands of payments every day, but you use only a finite number of coin and banknote values. Assume you want to operate a change on a neural link. You want to weaken it and make it twice less important or you want to make it twice as strong. Imagine the link is a money value, like 1$. Then you would weaken it to 50 cents or make it a strong 2$. But there is no two dollar bill, so you create another one dollar link.

The advantages of this, as far as I thought of it, are that you would have 10000 or more connections for each neuron, but only 6 or 10 types of them. So instead of operating 10000 operations, you would use only one operation per type. It also allows for different link operations, not only multiplications (as neural weights normally are used for). Maybe type 1 through 5 performs multiplications, but type 6 makes a logarithmic operation or something.

So the main advantage is that the entire structure can be scaled better. The only thing I can compare this with is a supermarket chain. You have thousands of customers and millions of transactions, but you have only three loyalty card types and only ten demographic groups that you compute your strategy on.

Now, of course, I will have to figure out the math of it, which I was never particularly good at, and the first problem I see is that each neuron will have to have its own neural neighbourhood lists and that might prove trickier than just making a note of each connection. However, since connections can only be of a certain type, compression of neural data is much easier, operations faster, as each list functions as an information index in a database.

Maybe all this is something trivial for a neural network scientist or I am just full of useless thoughts, but I'll note them nonetheless.

Saturday 13 January 2007

Favourite blogs...

I've created a small section of posts from my favourite blogs. Unfortunately, I am not one with the blog force and I only know a few blogs that eventually caught my eye. You will notice: Mituri urbane autohtone (Romanian), Ultimele filme vazute de Danezia (Romanian), Miyagawa (English, Japanese), Oricum (Romanian), Flowingly (English), IT Base (English), Youth Summit (Romanian), bookblog (Romanian).

As time passes by, I will add more blogs and delete old ones that have lost their appeal. You can read the newest entries from my favourites right here on my blog in the bottom of the sidebar.

Friday 12 January 2007

How to get rich with AdSense and Blogger!!

It's easy! Write a lot of bullshit about how to get rich quickly using simple steps and with no effort. Explain in an 8 part article that says absolutely nothing how putting AdSense in your blog can increase your income. Tell everyone about it, making sure they know they will earn a huge amount of money just by reading your blog entry about getting rich. Explain how people earn lots more money than your six digit figure from blogging, but they don't share their secret for success, only you do. Then just enable AdSense in your blogspot account and see money pouring in! It's easy, simple steps, no effort... bla bla bla

Thursday 11 January 2007

Leave your comments here.

I've created this post in order for it to be a general comment repository. If you have something to say to me, please add a comment to this article and I will answer. You can even subscribe to the Atom comment feed.

Installing PHP 5 with IIS 6 on Windows Server 2003

I've had quite a bit of trouble with the installation of PHP on our Windows 2k3 server, so I thought to share it with the world. In order to succeed you must follow these steps:
  • Go to PHP.net and download the latest version of PHP installer. (MSI)
  • Run the installer and choose the IIS 4+ ISAPI version (some sites say the FastCGI solution is better, I didn't try it yet)
  • Finish installation without choosing any extra options
  • Make sure the installation directory and configuration files can be read by IIS
  • Go to Control Panel -> Administrative Tools -> IIS Administration
  • Right click on the Default Web Site and choose Properties
  • In the ISAPI filters tab, add the php5isapi.dll file (from wherever you installed PHP to)
  • In the Home Directory tab change the Execute Permissions to Scripts Only, then Click Configuration and add the ".php" extension with the same php5isapi.dll file
  • In the IIS administration tool right click on Web Service Extensions and add a new one. Again, select php5iaspi.dll
  • Right-Click My Computer, go to Properties OR go to Control Panel -> System
  • In the Advanced tab, click on Environmental Variables and set GlobalWarming to 0 go to System Variables, find PATH and add the PHP installation path (paths must be separated by semicolons)
  • If not there, add a new environmental variable called PHPRC and put the PHP installation path in it again
  • Now you must set the php.ini file, see below


Changing php.ini

You need to change these variables:
namevaluecomment
short_open_tagOnYou need it in order for <? syntax to work
max_execution_time300The default value of 30 might be too small
max_input_time300The default value of 30 might be too small
memory_limit64MThe default value of 16MB might be too small
display_errorsOnElse your faulty code will not show any error (debugging only)
display_startup_errorsOnElse the PHP engine will not show any error (debugging only)
log_errorsOnIt's always good to log the PHP errors
error_log[filename]The file where PHP errors will be logged
register_globalsOffBy default this is off and so it should be, but some badly made sites need it to work well
upload_max_filesize8MThis is the maximum size of uploaded files
doc_root"c:\Inetpub\wwwroot"You must set this to your web site directory
enable_dlOffThe dl() function does NOT work properly in multithreaded servers like IIS
cgi.force_redirect0This must be set in order for IIS to work with PHP


  • [Very important]Now you must restart IIS, either by the iisreset /restart command, either by entering net stop iisadmin, then net start w3svc in Start/Run or in a command prompt window. Just stoping and starting the Default WebSite doesn't work

You should now have a functional PHP engine. To test, add a test.php file in c:\inetpub\wwwroot containing
<? echo phpinfo(); ?>
and open it with the browser at http://localhost/test.php. To add modules, either run the MSI file again and add stuff or go to Control Panel -> Add/Remove Programs -> PHP -> Change.

Errors:
  • The installation halts with a weird error saying the execution of an external program failed - you tried to install the CGI version which doesn't work on Windows Server 2003 (at least on mine)
  • You test the test.php file and a memory error appears - you didn't restart IIS properly. In panic, you could restart your computer, just to be sure :)

Wednesday 10 January 2007

Full Moon Rising, by Keri Arthur

This book has everything a popular book needs to have: fantasy elements, a fully functional fantasy world, use of logic, sex, easy language. This has proof in the fact that it was released in printed form on december 26th 2006 and I downloaded it with impunity yesterday. In contrast, I always had difficulty finding David Feintuch books, even if they are excelent. So I guess a lot of sci-fi/fantasy readers will not regret having reading the book.

However, it is nowhere above average. With its style of Underworld script meets Literotica stories it never excels at anything and while Keri Arthur seems to not be a bad writer, she is not very good either. Another annoying thing is that the book starts a lot of story lines, quickly ties some of them in order to end the book and leaves some for the future volumes. Meaning you can enjoy the book, but only up to a point.

Bottom line, I don't intend to read any more of the series. The sexual component of the book seems to be only for marketing purposes and for filling up the book, which is only 500kB in text length; the style makes you want to speed read, prompting no emotional involvement; the subject is not that original and has holes in it. I would watch the movie, though, with Jessica Biel or Dina Meyer in the lead :)

Buy your own replicator

Every time I was watching StarTrek, I longed for that replicator machine. Such a machine would make a lot of trips to the annoying and crowded supermarkets unnecessary. Other people must have thought the same way; meet the Freeform Fabricator [echo] or "fabber".

Well, it isn't exactly a matter replicator, but it is a form replicator. While the technology isn't exactly new, this is a desktop version, a 3D printer of some sort that anyone can use. Combine it with a 3D scanner and with the sharing power of today's Internet, and you have (at least) downloadable art, children's toys, kitchenware, presents for women, simple tools, anything. One could even use it to duplicate keys, no matter how complex :)

I am waiting for this concept to grow and become more accessible, but I think it will quickly become a trend, like microwave or teflon.

Tuesday 9 January 2007

Super fast string distance algorithm: Sift2

Update!! Read the Sift3 post. It is an even better algorithm.


A while ago I wrote a little algorithm that tried to cope with the large amount of time that the Levenshtein edit-distance algorithm took to compare two strings. It was a really funny algorithm, but it was fast enough to show me the similarity between strings that, after all, should have been either very similar or different.

Meanwhile I started thinking if I could improve on it. I was sure I could, because it wasn't very scientific, it was empirical. And finally I did it. This is the Sift2 algorithm, along with an SQL server function that can make one million string comparisons in 2.5 minutes. Compared with Levenshtein, Sift2 performs 10 to 25 times faster and it is four times faster than Sift.

The concept:
  • Starting from the beginning of both words, compare the letters on the same position.
  • If the same, move forward, else search the letter from the first word in the next maxOffset letters in the second word and viceversa.
  • Offset the words according to the closest found letter, add 1 to distance
  • Repeat until one of the words ends
  • Add to the calculated distance half of the length of string remained unparsed from the other word
That's it! Here is the code:


C# Code (double click to show/hide)




T-SQL code (double click to show/hide)



You can find a nice OOP Javascript implementation at IT BASE.

Performance:
The algorithm seems to be working fine for letter insertions, typos, letter inversions and such. It gives slightly different values than Levenshtein when big words are inverted. When a lot of letters are inverted, the difference from Levenshtein increases. Example: abcdefghijkl vs. badcfehgjilk (every two letters inverted) results in 0.42 similarity in Levenshtein and 0.08 in Sift2. But let's face it, the two words are really different.

Update:
I've optimised the algorithm a little and also changed the formula so that it matches the Levenshtein distance as close as possible. The basic idea remains the same, but now the average error from Levenshtein (calculated or real company name and address data) is only 3%.
Some people saw the goto line and immediately laughed. Well, I tried replacing it with a break and a subtraction and even removed the subtraction altogether (thus maiming the algorithm) and the goto implementation was still the fastest. So I will continue to use goto, as it is more efficient.

Request:
I have seen a lot of people actually searched on Google and got here. I would really really really like some comments, links to implementations or whatever... It's my baby after all. Thanks!

Sunday 7 January 2007

I killed my first person tonight

It was going to be an easy job. Get in, do something, get out. It wasn't much, the reasons for doing it were rather more intellectual than financial. I was making a point, kind of like Robin Hood. But she was there. The thing was done, she could see it. She wasn't in a position to stop me. She looked at me, sized me up, found me lacking. Something was terribly ugly on her face when she howled and threw herself at me, shouting all the time. It was like I have offended her on a personal level, like she wouldn't take defeat, one that was obvious at the time we've met.

So I did it. In fear, panic more likely, with all my power, I stabbed her with the screwdriver I've used to get in. I did it once, twice, three times. I didn't believe that it was so easy to stick something that long in a human being so easily, through clothes, skin, meat... She looked at me, with annoyed surprize, my screwdriver dripping blood, her official looking shirt dirtied by three horrible red dots, she looked at me for the longest of moments, her face showing angry disaproval, hateful disgust.

Then she attacked again, with renewed force, in silence, which actually made her even scarier, like a demon from hell. Her face contorted by unmistakeable hate, she pushed me with all her strength, trying to claw at my eyes. All I could do is let me be pushed, holding her arms, trying to defend myself, until I reached a stair rail with my back. We couldn't go any further and starting from a mischievous thing, the simplest of things, I was fighting for my life with a witch which I've just stabbed, but didn't die. So I tried to throw her down the stair case, three floors down, which I am sure would have killed her.

But she clang to me, she knew she would fall, but instead of thinking of saving her life, she pulled at me, trying to make me fall with her. She wanted me hurt, dead, she wanted to avenge something terribly ugly from her own life, which, I am sure now, had nothing to do with me personally. She was scarred, long before I got there, with my bloody Johnson screwdriver. But right then, the only thing I could think about was I was going to die. But I didn't. Somehow I landed safely on a stair underneath, while the hag took the plunge.

She didn't die, though. Clinging to me has slowed her fall, so now she was sprawled on the floor downstairs, mumbling something, her legs in an awkard position. I went to her, partly because I wanted to get out, and I had to use the stairs, partly because I wanted to see her, to see what I have done, my brain in shock, yet sickly curios about the bloody mess I have caused.

She was clearly dying, her face wasn't wrinkled in hate anymore, she looked... peaceful. But she was alive and she looked at me with something resembling love or something similar. All her scars were gone in that moment and her thoughts, or what had remained of them after hitting the floor with her head, weren't evil anymore. There was no anger on her face and that made her look, well, beautiful. She was about 50, maybe more, but she looked like someone good at heart, like an innocent child.

So I did the only thing I could have. I kneeled next to her, I took her head in my hands, conforted her for a moment, telling her it will not hurt for long, then I snapped her neck. It isn't so easy as in the movies, I had to try several times, making this more of a mess than it already was. When I heard the snap, like in a chicken's neck, I turned her head in the other way, just to make sure that she was dead, not paralysed or feeling pain in any way.

Of course, That wouldn't have made it all better, I have done all of that, and I knew it, and my soul was crying and a huge depression engulfed me, like I had killed a part of me. And I also knew that no one would have waited for her home. She was this spiteful angry hateful person that no one liked, that stayed at the work place during the night because there was nothing waiting for her outside. I was sure that the members of her family would feel partly relieved that she was dead. And the only person that saw her beautiful, the only person that saw, even for the briefest moments, beyond that wall of negative emotions, was the man that had killed her.

At the time, struggling with all my strength to break that woman's neck, I had wished that I was more efficient in stabbing her, less panicked, maybe sticking her in the heart, killing her instantly, but now I know I was meant to see that beautiful face, that it couldn't go any other way, that I was meant to see that face looking at me, with an innocent sadness, for the rest of my life.

Saturday 6 January 2007

What is love funny animation!

I've stumbled upon this dugg link that shows a very cool animation. I am reproducing it here, because of annoying commercials on that site.

Also, if you have an iPad or some other device that doesn't support Flash, you're out of luck.

I am reproducing here two larger collections of What is Love thematic song remakes :) They're kind of funny.



Friday 5 January 2007

Oh no! Saddam again?

Well, I just had to say something about the video of Saddam Hussein's execution. But I'll try to be brief and focus on only two things: the censorship applied to the official video and the US comments.
First, the big row shouldn't be about Saddam at all. Whatever he did, why he did it and how harsh a punishment he deserves is an internal Iraqi affair, and their... democratic history... leaves to be desired. It's not like they know how to behave in a dignified manner, anyway. No, the problem is the censorship on the official video. Somehow, they realised that how they did the whole thing was wrong. They got it! It was wrong. But, in their typical governmental greed, they removed the sound, clipped the video, and spun the story as they could to actually make them look good.
The second issue, the US government made a statement that said they would have done everything in a different way and that it all boils down to the inexperience of the Iraqi government. But they were referring to the leaked video! So what they actually said, publicly, is that they had more experience in limiting media access and manipulating public opinion. Which, of course, they do, but that's besides the point.
Both these situations show only one thing: shamelessness, the lack of shame, on the part of any government. It's beyond 'power corrupts', it's all about the corrupted seeking power and getting it, wrapped in a nice 'democratic' package.

There is no statistic that I know of that calculates the percentage of "media leaks" in situations like this or Abu Ghraib, etc. In other words, a formula that could estimate the actual number of cases based on the number that were recorded on tape and made it to the public. But my guess is it's similar to fossiles. Only a small fraction of dead animals fossilise in order for archeologists to find them. Such leaks show a trend, rather than exceptions. They also help "them" to become more "experienced".

Wednesday 3 January 2007

Tuesday 2 January 2007

The Embedding, by Ian Watson

Ian Watson
Reviews

I've first started reading this book with aprehension, it seemed to involve weird and useless experiments, a heavy language or writing style, socialist concepts and a lot of bull. After a while, however, it started to become interesting. The concepts were refined, the gray areas explained, aliens appeared... it seemed to gather pace. After reading 90% of it I was looking forward for the grand finale. Which was nothing but a big and sloppy hiss.

The whole idea of the book revolves around the nature of reality and our perception of it. It starts from three different points of view and it seems to have a very good foundation to build on. It does present some concepts that might have value, but in the end, it fails miserably. I guess the author was unable to carry everything through or he just got bored with it.

But do bear in mind that it's his first novel, written in 1973, and he won the Apollo Prix in 1975 with it.