Request.Url and the missing port

We have this web application that needs to call a third party site that then redirects back to us. The other app is using a configured URL to redirect back. In order to develop and debug the application, we used a router redirect with a different port like this: the external site calls http://myExternalIp:81 and it gets redirected to my own computer on port 80.

I was amazed to notice that when entering my local page, Request.Url would be in the format http://myExternalIp, without the 81 port. As the page was executed in order to debug it, I was baffled by this behaviour. I tried a few things, then I decided to replicate it on a simple empty site and there it was. The only thing I could find that had any information about the original port number was Request.Headers["Host"] which looked something like myExternalIp:81.

I guess this is a bug in the Request object, since it uses the port of the actual server instead of the one of the request, since my server was responding on port 80 on localhost and not 81.

Here is a small method that gets the real Request URL:

 public static Uri GetRealRequestUri()
 {
     if ((HttpContext.Current == null) || 
         (HttpContext.Current.Request == null))
         throw new ApplicationException("Cannot get current request.");
     return GetRealRequestUri(HttpContext.Current.Request);
 }

 public static Uri GetRealRequestUri(HttpRequest request)
 {
     if (String.IsNullOrEmpty(request.Headers["Host"]))
       return request.Url;
     UriBuilder ub = new UriBuilder(request.Url);
     string[] realHost = request.Headers["Host"].Split(':');
     string host = realHost[0];
     ub.Host = host;
     string portString = realHost.Length > 1 ? realHost[1] : "";
     int port;
     if (int.TryParse(portString, out port))
         ub.Port = port;
     return ub.Uri;
 }

Read More

(Response.Redirect or Response.End) and 'Thread was being aborted' error

Just a short infomercial. Response.Redirect(url) is the same with Response.Redirect(url,true), which means that after the redirect, Response.End will be executed. In case you get a weird 'Thread was being aborted' exception, you probably have the Redirect/End methods inside a try/catch block. Remove them from the block and it will work. Probably the ending of the Response session doesn't look good to the debugger and that particularily obtuse exception is thrown.

If you absolutely must put the thing in a try/catch block, just put everything EXCEPT the Redirect/End. Another option (only for Response.Redirect) is to add a false parameter so to not execute Response.End.

Read More

Microsoft joins the Apache Foundation

And still, pigs are not yet flying! Here is the news article:
history.forward()

Bottom line: IIS is being optimized for PHP, PHP is being optimized for Sql Server and, by becoming a sponsor of Apache, Microsoft probably gets a say in their development.

What a typical corporate move that is. But what would be the effect of such a move only time will tell. Will ASP.Net start working habitually on Apache?

Read More

SQL Best Practices

This post will be quite lengthy and it will detail my findings on best practices with SQL, specifically Microsoft SQL Server.

I started with an article written by Vyas Kondreddi in 2001: SQL Server TSQL Coding Conventions, Best Practices, and Programming Guidelines. In 2001 people were not microblogging!

Well, to summarize the article and bring it up to date a little, here are some of the most important points (in my view):

• Decide upon a database naming convention, standardize it across your organization, and be consistent in following it. It helps make your code more readable and understandable.
• Write comments in your stored procedures, triggers and SQL batches generously, whenever something is not very obvious.
• Try to avoid server side cursors as much as possible.
  As Vyas Kondreddi himself says: "I have personally tested and concluded that a WHILE loop is always faster than a cursor"
• Avoid the creation of temporary tables while processing data as much as possible, as creating a temporary table means more disk I/O. Consider using advanced SQL, views, SQL Server 2000 table variable, or derived tables, instead of temporary tables.
  This is interesting, because I usually use a lot of temporary tables in my stored procedures to make the code more orderly. I guess that in the case of SQL Server 2005 and later one can always use Common Table Expressions to make the code more readable. For SQL 2000 and such I found two interesting articles about not using temporary tables and replacing them with either derived tables (selects in selects) or with table variables, although they do have some limitations, thoroughly explained in the latter post. Here are the links: Eliminate the Use of Temporary Tables For HUGE Performance Gains and Should I use a #temp table or a @table variable?
• Try to avoid wildcard characters at the beginning of a word while searching using the LIKE keyword, as that results in an index scan, which defeats the purpose of an index.
  For a short analysis of index scans go to SQL SERVER - Index Seek Vs. Index Scan (Table Scan).
• Use the graphical execution plan in Query Analyzer or SHOWPLAN_TEXT or SHOWPLAN_ALL commands to analyze your queries.
• Use SET NOCOUNT ON at the beginning of your SQL batches, stored procedures and triggers in production environments, as this suppresses messages like '(1 row(s) affected)' after executing INSERT, UPDATE, DELETE and SELECT statements. This improves the performance of stored procedures by reducing network traffic.
• Use the more readable ANSI-Standard Join clauses instead of the old style joins.
• Incorporate your frequently required, complicated joins and calculations into a view so that you don't have to repeat those joins/calculations in all your queries.
• Use User Defined Datatypes if a particular column repeats in a lot of your tables, so that the datatype of that column is consistent across all your tables.
  Here is a great article about Sql UDTs (not the new .NET CLR types): What's the Point of [SQL Server] User-Defined Types?. Never used them, myself, but then again I am not an SQL guy. For me it seems easier to control data from .Net code
• Do not let your front-end applications query/manipulate the data directly using SELECT or INSERT/UPDATE/DELETE statements. Instead, create stored procedures, and let your applications access these stored procedures.
  I am afraid I also fail at this point. I don't use stored procedures for simple actions like selecting a specific item or deleting a row. Many time I have to build search pages with lots of parameters and I find it really difficult to add a variable number of parameters to a stored procedure. For example a string that I have to split by spaces and search for all found words. Would it be worth to use a stored procedure in such a situation?
• Avoid dynamic SQL statements as much as possible. Dynamic SQL tends to be slower than static SQL, as SQL Server must generate an execution plan every time at runtime.
  Personally, I never use dynamic SQL. If I need to create an SQL string I do it from .Net code, not from SQL.
• Consider the following drawbacks before using the IDENTITY property for generating primary keys. IDENTITY is very much SQL Server specific, and you will have problems porting your database application to some other RDBMS. IDENTITY columns have other inherent problems. For example, IDENTITY columns can run out of numbers at some point, depending on the data type selected; numbers can't be reused automatically, after deleting rows; and replication and IDENTITY columns don't always get along well.
  So, come up with an algorithm to generate a primary key in the front-end or from within the inserting stored procedure. There still could be issues with generating your own primary keys too, like concurrency while generating the key, or running out of values. So, consider both options and go with the one that suits you best.
  This is interesting because I always use identity columns for primary keys. I don't think a data export or a database engine change justify creating a custom identity system. However I do have to agree that in the case that data is somehow corrupted a GUID or some other identifier would be more useful. I am sticking with my IDENTITY columns for now.
• Use Unicode datatypes, like NCHAR, NVARCHAR, or NTEXT.
• Perform all your referential integrity checks and data validations using constraints (foreign key and check constraints) instead of triggers, as they are faster.
• Always access tables in the same order in all your stored procedures and triggers consistently. This helps in avoiding deadlocks. Other things to keep in mind to avoid deadlocks are: Keep your transactions as short as possible. Touch as few data as possible during a transaction. Never, ever wait for user input in the middle of a transaction. Do not use higher level locking hints or restrictive isolation levels unless they are absolutely needed. Make your front-end applications deadlock-intelligent, that is, these applications should be able to resubmit the transaction incase the previous transaction fails with error 1205. In your applications, process all the results returned by SQL Server immediately so that the locks on the processed rows are released, hence no blocking.
  I don't have much experience with transactions. Even if I would need transactions in some complex scenarios, I would probably use the .Net transaction system.
• Offload tasks, like string manipulations, concatenations, row numbering, case conversions, type conversions etc., to the front-end applications.
  Totally agree, except the row numbering, where SQL 2005 added all those nice Getting the index or rank of rows in SQL Server 2005 aggregate ranking options
• Always add a @Debug parameter to your stored procedures. This can be of BIT data type. When a 1 is passed for this parameter, print all the intermediate results, variable contents using SELECT or PRINT statements and when 0 is passed do not print anything. This helps in quick debugging stored procedures, as you don't have to add and remove these PRINT/SELECT statements before and after troubleshooting problems.
  Interesting, I may investigate this further, although the SQL debugging methods have improved significantly since the article was written.
• Make sure your stored procedures always return a value indicating their status. Standardize on the return values of stored procedures for success and failures. The RETURN statement is meant for returning the execution status only, but not data. If you need to return data, use OUTPUT parameters
• If your stored procedure always returns a single row resultset, consider returning the resultset using OUTPUT parameters instead of a SELECT statement, as ADO handles output parameters faster than resultsets returned by SELECT statements.
• Though T-SQL has no concept of constants (like the ones in the C language), variables can serve the same purpose. Using variables instead of constant values within your queries improves readability and maintainability of your code.

The next stop was SQL Server Best Practices from Microsoft.

Here are the articles I found most important, covering stuff from testing the I/O system of the system you want to install SQL server to up to Database backup, mirroring and maintainance:
Predeployment I/O Best Practices
SQL Server 2005 Deployment Guidance for Web Hosting Environments
SQL Server 2005 Security Best Practices - Operational and Administrative Tasks
Comparing Tables Organized with Clustered Indexes versus Heaps
Troubleshooting Performance Problems in SQL Server 2005
Implementing Application Failover with Database Mirroring
SQL Server 2005 Waits and Queues
TEMPDB Capacity Planning and Concurrency Considerations for Index Create and Rebuild
The Impact of Changing Collations and of Changing Data Types from Non-Unicode to Unicode
XML Best Practices for Microsoft SQL Server 2005
Performance Optimizations for the XML Data Type in SQL Server 2005
Top 10 Hidden Gems in SQL Server 2005

Lastly some links that I will not go in depth on:

SQL Server 2000 Best Practices
SQL SERVER - 2005 Best Practices Analyzer Tutorial - Sample Example describes the Microsoft Best Practices Analyser application. I tried it myself, it's not much. It touches mainly on the maintainance and security issues that I don't really concern myself with.
Top 10 Best Practices for Building a Large Scale Relational Data Warehouse. I don't think I will need it soon, but it is a short and interesting read.
SQL Server Pre-Code Review Tips. This Pinal Dave guy is pretty cool. He seems like a good resource for SQL related issues.
CMS Database Administration SQL Server Standards, a set of SQL coding standards for a medical government agency.

I am sure there are a lot of interesting resources on the net. I will update this post with new information once I get to it.

Read More

Rebellious Computer

My computer was never the sanest of them all. Its most peculiar problem was that, after running for a while, you could not reset it. You had to either lower the CPU frequency or shut it down, wait for it to cool down, then start it up. And funnily enough, it did that only when having 512Mb of RAM. With an additional 512Mb chip it did not present this abnormality.

About two weeks ago, it started to show "Write delayed failed" errors during the night, when the only utility running was some file sharing app. I started researching and found that the error itself wasn't indicative of anything! A lot of people had the same problem, but each and one of them for different reasons. It was like an "I am on strike" sign from my computer.

Well, I thought updating drivers for my SATA hard drive may solve the problem. In the same swoop I also updated to the latest NVidia display drivers. Nothing changed. I then looked for an application that tells me what is the temperature of my CPU. I found SpeedFan, which is completely freeware and which I highly recommend, application that indicated a temperature of about 63 Celsius.

I thought it was too much so I've decided to clean up my CPU cooler. Between the cooler fan and the metal radiator there was a 2 mm thick dust cover. No wonder it was so hot. I cleaned everything up and my temperature dropped to about 51C. Glad I'd solved everything I started playing a game. Nothing fancy, not something 3D or anything complicated. The computer suddenly reset itself.

Now that was a complete revolt! Frustrated I started a movie and went to the other room to watch it on my TV. You see, I use this wireless AV connector to transfer the TVout signal from my display adapter to the TV in the other room. I could hear everything, but the screen was blank. What the...?

After hours of trial and error I've reached the conclusion that the newest drivers just couldn't do video overlay on the TV. Also, the fact that I chose 1280x1024 as my monitor resolution and it showed in 640x480 was also from the drivers. Funny enough, when I removed the "clone to TV" option, it worked ok. The solution was to download and install an older NVidia driver, version 91.47. When everything was said and done and I was half in watching the movie, the computer reset itself!

I am now at the stage where I found an overclocking setting on the NVidia drivers and when fiddling with it, I get my computer to reset. It is funny that it does that even if I set a lower clock setting, which was my original intention anyway. I believe that the new drivers (and the new not so new drivers) are making my NVidia FX 5200 card go too fast. It overheats and causes my computer to stop. I can only hope that buying a cooler fan for the video card (it has only a small metal radiator) will solve the problem.

Meanwhile, I have set the Hardware acceleration setting from the 3D Settings menu of Nvidia control panel to Single display performance mode instead of Multiple display performance mode as it was set, since I have only one monitor and a TV clone. I could play that game without a reset, although that is not really indicative of anything yet.

Please, computer, I am sorry! I will never do it again! Please forgive me! Don't act insanely anymore! :(

Update: in the end it was neither the video card nor the processor. It was the SATA RAID drivers!! Updating them was NOT a good idea. I went to the specific site of the motherboard I have and installed some drivers from 2003 and it now works without any glitch.

But how did I realize that was the source? Because in a rare stroke of luck I watched the computer display a blue screen and then resetting itself. In Windows XP, if you go to My Computer - Properties - Advanced - Startup and Recovery you find a System Failure groupbox with a bunch of options. Unchecking the "Automatically restart" box will make the computer display the BSOD and NOT reset, giving you the opportunity to read the error message. In my case it was a viamraid.sys giving the error 0x000000D1: VER_IRQL_NOT_LESS_OR_EQUAL.

Update 2: The problem with Write Delayed Failed was not from the SATA drivers, but from the USB external hard drive. After trying everything I knew and found on Google, I was really pissed off that this still happened, until I ran into an obscure forum where a guy said his problems all went away after changing the physical USB port!! So I moved the actual USB jack and the problem was solved!

Some other explanation I found were about the protocol of sending data packets. First it is sent a small chunk of data then, if everything went ok, the next data chunk is twice as big, so 256kb, 512, 1024, 2048 and so on. Windows has some problems with sending packets bigger then 1024Kb! There is a utility on the net that patches a Windows system file to address the issue. I might have run it, I don't even remember it's name :) I pretty much threw everything at my computer and ran it for a few weeks until I was satisfied that it worked. But do try the USB port change first...

Read More

Pushing files from Javascript

How can I get some content from javascript (like a string) and send it to the client as a file? I don't have access to the content I want to send from the server.

This was a question posed to me in a rather more complex way: how do I take some file content from a web service and send it to the client as as file without downloading the content to the web server first?

The simple answer right now: you cannot do it. If you guys know more about this, please let me know. I've exausted all avenues I could think of, but then again, I am no master of Javascript and html responses.

Here is what I have tried. Basically, I have a string like a html table and I want it sent to the client browser as an excel download. So I opened a new window with javascript and tried to write the content there:

var win2=window.open('');
win2.document.write(s);

It worked and it displayed a table. Now, all I wanted to do is add/change the html header content-type to application/vnd.ms-excel. Apparently, you can't do it from Javascript. Ok, how about getting the necessary headers from the ASP.Net server? (remember, the restriction was that only the file content should not come from the web server). So I created a new page that would render a completely empty page with the right headers:

protected void Page_Load(object sender, EventArgs e)
    {
        Response.Clear();
        Response.Buffer = true;
        Response.ContentType = "application/vnd.ms-excel";
        Response.AppendHeader("Content-Disposition", "attachment;filename=test.xls");
        Response.Charset = "";
        
        Response.Flush();
        Response.End();
    }

Then I just opened it in a new window (just ignore the browser pop-up filters for now) with

var win2=window.open('PushFile.aspx');
win2.document.write(s);

What happened was that the page was just rendered like a normal page. How come? I change the code so that it would write the content after a few seconds. And I got this: first the browser asks me if I want to permit downloading the file, then, after a few seconds, the warning goes away and the string is displayed in the new window. I tried with document.createTextNode, it didn't work.

So far, none of my attempts to serve javascript content as a binary file worked. If you know of a way to achieve this, please let me know. Thanks!

Update:
Meaflux took a swipe at this request and came up with two delicious ideas that, unfortunately, don't really work. But I had no idea things like these existed, so it is very much worth mentioning.

First: the data URI. Unfortunately it is only supported by FireFox and such and has no way of setting a content-disposition header or some other way of telling the browser that I actually want it saved. It would work for an excel file, but an image, for example, would be opened in a browser window.

Second: the IE execCommand javascript function which has a little command called SaveAs. Unfortunately this would only work for actual HTML pages. Even if the browser would open a binary file, I doubt that a saveAs command would save it correctly.

Besides, both these options, as well as my own attempts above, have a major flaw: there is no way to send chunks of data as you are receiving them from the web service. What is needed it declaring some sort of data stream, then writing stuff in it and then declaring it programatically closed.

Read More

DataFormatString not working in GridView!

I've spent about half an hour trying to determine why the DataFormatString format would not be applied to cell values in a GridView. The short answer: set the HtmlEncode property of the BoundField to false!

Now for the long answer:

A while ago I wrote a small article about how to format the data in your autogenerated GridView columns. At the end of the post I added a small update that explained why I set the HtmlEncode to false. It was, in my opinion, a GridView bug.

However, I didn't realise at the time that the same thing applies to normal GridView BoundFields as well. The thing is, in order to display a value in a bound cell, it FIRST applies the HtmlEncoding to the value CAST TO STRING, THEN it applies the FORMATTING. Here is the reflected source:

/// <summary>Formats the specified field value for a cell in the <see cref="T:System.Web.UI.WebControls.BoundField"></see> object.</summary>
/// <returns>The field value converted to the format specified by <see cref="P:System.Web.UI.WebControls.BoundField.DataFormatString"></see>.</returns>
/// <param name="dataValue">The field value to format.</param>
/// <param name="encode">true to encode the value; otherwise, false.</param>
protected virtual string FormatDataValue(object dataValue, bool encode)
{
    string text1 = string.Empty;
    if (!DataBinder.IsNull(dataValue))
    {
        string text2 = dataValue.ToString();
        string text3 = this.DataFormatString;
        int num1 = text2.Length;
        if ((num1 > 0) && encode)
        {
            text2 = HttpUtility.HtmlEncode(text2);
        }
        if ((num1 == 0) && this.ConvertEmptyStringToNull)
        {
            return this.NullDisplayText;
        }
        if (text3.Length == 0)
        {
            return text2;
        }
        if (encode)
        {
            return string.Format(CultureInfo.CurrentCulture, text3,
         new object[] { text2 });
        }
        return string.Format(CultureInfo.CurrentCulture, text3, 
        new object[] { dataValue });
    }
    return this.NullDisplayText;
}

At least the method is virtual. As you can see, there is no way to format a DateTime, let's say, once it is in string format.

Therefore, if you ever want to format your data in a GridView by using DataFormatString, you should make sure HtmlEncode is set to false! Or at least create your own BoundField object that implements a better FormatDataValue method.

Read More

Code Geass: Lelouch of the Rebellion

I know, you're thinking "Who made the great effort of coming up with this incredible gay name?", but keep reading a little more, because this anime series is rather interesting. I am refraining from calling it cool, since the name and because it is not over yet and because it is partially mecha. Also, because I think the direction it is going is a bit off course. Now that the bad things are out of the way, let me tell you about the good ones.

Anyway, the whole thing revolves around Lelouch, the third prince in line for the throne of the Holy Empire of Britannia. It is set in an alternate universe where battles are fought with humanoid robots called Knightmares, and the above mentioned empire considers the Britannians first class citizens while any other enslaved nation gets a number that designates its teritory and its people. The Japanese are called Elevens after the occupation of Japan, and Japan itself is renamed to Area 11.

I will let you read the plot in the Wikipedia page, and focus on the good bits: Lelouch is a very smart guy, he plays chess and defeats just about everybody. He uses his strategic skills to fight against the empire of Britannia as the faceless terorist Zero - for reasons too complicated to explain here. He is still in highschool (why must every Japanese story happen in high schools?!) and he has one more advantage: a geass. This is a magical ability that allows him to command any person he has eye contact with.

The first season had 25 episodes and was pretty cool. It involved strategy, drama, action, sci-fi and a tight script. The second season (R2) is more complex, but my opinion is that it lost much of the power of the first season and has reached episode 14 (released with English subs today). It is worth mentioning that the team that made Code Geass also worked on Planetes, a sci-fi anime based on Arthur C. Clarke's ideas, which I also liked a lot.

Some links:
Code Geass Wikipedia page
Bandai Entertainment Code Geass page
Watch Code Geass online

Read More

Transcendent by Stephen Baxter

Oh, no! After such a glorious second volume, Baxter regressed for the third volume of the Destiny's Children series, Transcendent. What you get is basically a continuation of the first volume, but without the emotional content or the cool ideas of Coalescent. Same awkward family relationships that no one really cares about, same main character who is actually driven by the actions and thoughts of people around him, rather than his own, same single final moment that shapes the world without actually making the reader feel anything, same lengthy dialogue that brings important issues into discussion, but without drawing the reader in.

As Stalin said, one death is a tragedy, one million is a statistic. Same thing applies to humans 500.000 years into the future, going back into the past to redeem the sins of humanity. No one cares! The Earth is pushed to the edge by global warming and the lead character is championing a great hydrate stabilisation engineering project. Who cares?!

Bottom line: the book was well written, but badly designed. It's like an engineer doing a great job building something that is fundamentally flawed. I struggled to finish the book just as I've struggled to finish Coalescent, which was far more interesting to begin with. The reason is simple: the reader cannot really empathise with any of the characters, except in disparate fragments of the storyline.

Read More