Artemis, by Andy Weir

The Martian was a total surprise as it took the world by storm and became beloved by all science, tech and fast talking pun lovers in the world. Artemis, on the other hand, will not be. Andy Weir writes another book about a supersmart tech savvy character with a great sense of humor, this time solving technical problems on the Moon. Is it as good as The Martian? I don't know, because now I have a template and knew what to expect. Without the element of surprise, it's difficult to compare the two books. Yet I finished reading the novel in a day. That's gotta count for something.

Stylistically it is first person, filled with dialogue and action. It's incredibly easy to read. This time, the main character is a young woman and the location is a city on the Moon. Andy Weir is nothing if not optimistic, so in his book we do get there. I thought the action was following the pattern of danger-action-solution too closely, so much in fact that at one time I saw the entire story as a big adventure game. I will bet you lots of slugs that Weir loved Sierra games as much as the next geek (that being me).

Bottom line is that I liked the book, I gobbled it up and enjoyed it to the end. I didn't like Jazz Bashara as much as Mark Watney and, while the technological descriptions kept me interested, I still think that space mechanics and Martian farming trump EVA shenanigans and vacuum welding any time. It was still damn entertaining, though.

Read More

Oryx and Crake (MaddAddam #1), by Margaret Atwood

All in all I liked Oryx and Crake. Margaret Atwood is writing in a very technical manner, at least it felt like that to me, with scenes and characters constructed in a certain way. That is both instructional and a bit off putting, because once you become aware of it, it distracts from the plot. The plot itself is simple enough: the world has ended and there is this half crazed old man who is reminiscing over what happened. At first it is all unnervingly vague, with things being kept from the reader as the feelings and mental issues of the main (and single) character of the book are being declared. Then, when vagueness cannot be sustained anymore, it is all told as a linear story of what happened, with clear explanations of everything. No mystery left.

To me, it felt like Atwood wanted to warn about the way we treat our world and she just exaggerated some things while letting others be whatever she wanted them to be. From a scientific and sociological standpoint some of the things described are spot on, others are complete bogus. I would go into details, but I don't want to spoil the story. One example is the marketing brands that she cooked up and that are all (and I mean all) phonetically spelled descriptions or sometimes puns. All companies, all product names, all new animal names are like that and it gets really tiresome after a while. Also people are able to do wonderful (or horrible) new things in some areas and not advance at all in others. The timeline is a little screwed up, too.

My favorite quote, the one that made me think the most, was: "All it takes," said Crake, "is the elimination of one generation. One generation of anything. Beetles, trees, microbes, scientists, speakers of French, whatever. Break the link in time between one generation and the next, and it's game over forever" . He is referring to technology after a post apocalyptic event. There are few people that understand any instructions left by makers of advanced technology and the ones that can are ill equipped for survival.

The book does feel like it was written as standalone, with the possibility of being continued, which is a good thing: no ending cliffhangers and too much setting up the next books. The title is weirdly misleading, though, as both Oryx and Crake appear very little in the memories of the main character. Why was the book named so? It makes little sense. I am not sure if I will be reading the next books in the MaddAddam trilogy.

Bottom line: good, although a tad bland writing. A single character reminiscing is all that happens in the book. Grand ideas that most of the time make sense, but sometimes fall flat. Scientifically things don't go as she described.

Read More

Analyzing memory leaks in .NET applications running in production

Intro

I had this situation where an application that was running in production was getting bigger and bigger until it would devour all memory. I had to fix the problem, but how could I identify the cause? Running it on my computer wouldn't lead to the same increase in memory, what I wanted to see was why a simple web API would take 6Gb in memory and keep growing.

The process is simple enough:

1. Get a memory dump from the running process
2. Analyze the memory dump

While this post relates specifically to process memory dumps, one can get whole memory dumps and analyze those. There are two types of process memory dump formats: mini and full. I will be talking about the full format.

Fortunately, for our process, there are only two steps. Unfortunately, both are fraught with problems. Let's go through each.

Getting a memory dump from a running process

Apparently that is as simple as opening Task Manager, right clicking on the running process in the Details tab and selecting Create dump file. However, what this does is suspend the process and then copy all of its memory in a file in %AppData%/Local/Temp/[name of process].dmp. If the process is being watched or some other issue occur, the dumping will fail. Personally, I couldn't dump anything over 2Gb when I was trying to get the info from w3wp.exe (the IIS process) as the process would restart.

There are a lot of other ways of getting a process memory dump, including crash dumps, periodic dumps, event driven dumps. ProcDump, one of the utilities in the free SysInternals suite, can do all of those. The Windows Error Reporting (WER) system also generates dumps for faulty or unresponsive application or kernel. Other options for collecting a dump are userdum.exep, Windows Debugger (ntsd or windbg). Process Explorer, of course, can create two types of memory dumps for a process: mini and full.

Analyzing a process memory dump

This is where it gets tricky. What you would like is something that opens the file and shows a nice interactive interface explaining exactly what hogs your memory. Visual Studio does that, but only the Ultimate version gives you the memory analysis option. dotMemory from JetBrains is great, but it also costs a lot of money.

My personal experience was using dotMemory in the five days in which it is free and it was as seamless as I would have wanted: I opened the file, it showed me a pie graph with what objects were hogging my memory, I clicked on them, looked where and how they were stored and even what paths could be taken to create those instances. It was beautiful.

Another option is SciTech's .NET Memory Profiler, but it is also rather expensive. I am sure RedGate has something, too, but their trials are way to intrusive for me to download.

I have been looking for free alternatives and frankly all I could find is WinDBG, which is included in Windows Debugging Tools.

Windows Debugger is the name of the software. It is a very basic tool, but there are visualizers which use it to show information. One of them is MemoScope, which appears unmaintained at the moment and is kind of weird looking, but it works. Unfortunately, it doesn't even come close to what dotMemory was showing me. If analyzing a random .NET app dump showed that the biggest memory usage was coming from MainWindow (makes sense) which then had a lvItems that held most of memory (it was a ListView with a lot of data), WinDbg and therefore MemoScope show the biggest usage comes from arrays of bytes. And it also makes sense, physically, but there is no logical context. A step by step memory profiling guide using WinDbg can be found here.

Conclusion

I am biased towards JetBrains software, which is usually amazing, although I haven't used it a quite a while because their obnoxious licensing system (you basically rent the software). I loved how quickly I got to the problem I had with dotMemory. .NET Memory Profiler from SciTech is way cheaper and seems to point in the right direction, but doesn't really compare in terms of quality. Anything else I've tried was quite subpar.

Doing it yourself

As they say, if you want to do something right, you gotta do it yourself.

There is a Windows API called MiniDumpWriteDump that you can use programatically to create the dumps. However, as you've seen, that's not the problem, the dump analysis is.

And it's hard to find anything that is not very deep and complicated. Memory dumps are usually analysed for forensic reasons. There are specialists who have written scores of books on memory dump analysis. What's worse, most people focus on crash dumps, with memory leak analysis usually done while debugging an application.

I certainly don't have the time to work on a tool to help me with this particular issue. I barely had the time to look for existing tools. But if you, guys, can add information to this blog post, I am sure it will help a lot of people going through the same thing.

Pretty Pictures

I didn't find a nice way of putting the images in the post's text, I am adding them here.

Read More

MemoryCache.Default suddenly not working

MemoryCache has a really strange behavior where all of a sudden, without any warning or exception, it stops caching. You use Set or Add, it doesn't matter. It returns true for Add, it throws no error for Set, but the cache stubbornly remains empty. This behavior can be replicated easily by running this line:

MemoryCache.Default.Dispose();

From that moment on, MemoryCache.Default will not cache anything anymore.

Of course, you will say, why the hell would you dispose the default object? First of all, why give me the option? It's not like I can set the Default instance after I dispose it, so this breaks a lot of things. Second of all, why pretend to continue to be working when in fact it is not? Most disposable classes throw exceptions if they are used after being disposed. And third of all, it might not be you who does the disposing.

In my case, I was trying to be a good boy and declare all my dependencies upfront. I was using StructureMap as a IoC framework and for my class that was using a MemoryCache I declared a constructor parameter of the basest type I could: ObjectCache. Then I registered MemoryCache.Default as the instance used when needing an ObjectCache. The problem is that each unit test would initialize and tear down the StructureMap container and it would take all the instances used that implement IDisposable and dispose them, including our friend MemoryCache.Default. As I couldn't find a simple way to tell StructureMap to not dispose the object, I had to create a MemoryCacheWrapper that implemented ObjectCache and was NOT IDisposable and was using MemoryCache.Default for all methods and use that as the singleton instance for an ObjectCache in StructureMap.

If you are working with a .NET version prior to .NET 4.5, you may encounter the same issue with a bug in .NET: MemoryCache Empty : Returns null after being set

Read More

The Ends of the World: Supervolcanoes, Lethal Oceans, and the Search for Past Apocalypses, by Peter Brannen

This guy can write! I mean, Peter Brannen is writing about paleontology and climate change and the words just sing to you, make you see worlds that we actually know very little about and feel for ammonoids like there would be the cutest kittens, looking at you with big eyes and begging not to get extinct. As someone who wants to write himself someday, I really hate this guy. He is that good.

That being said, The Ends of the World is a book about the major extinctions in Earth's history, their causes and how they relate to our present lives. It's a captivating read, evoking lost worlds and very carefully analyzing how they disappeared and why. There is humor, drama, interesting interviews with fascinating people. Dinosaurs? Puh-lease. This takes you back to the good old days of the Ediacaran and slowly brings you around our time and even speculates on what could come after us, the hubris filled species that, geologically speaking, was barely born yesterday - or a few seconds ago, depending on how you measure it - and has the chance to outshine all the other species that came, ruled and went.

There is no way I can do it justice other than to say that I loved the book. In itself it's a summary of life on Earth so summarizing it myself would be pointless. I highly recommend it.

Here is the author, presenting his book at Google Talks:

Read More

Wix setup Error 0x8007007a: Failed to copy SqlScript.Script or Error 0x8007007a: failed to read SqlScripts table

You are trying to install an application and in the detailed log you get something like this:

InstallSqlData:  Error 0x8007007a: Failed to copy SqlScript.Script: SqlWithWindowsAuthUpv2.9.5UpdateGetInstallJobBuildByInstallJobBuildIdScript

or

InstallSqlData:  Error 0x8007007a: failed to read SqlScripts table

For me it was related to the length of an <sql:SqlScript> tag Id property. If you are trying to use more than 55 characters for the BinaryKey property, you get a build error that you should use a max of 55 characters. For the Id you get no such warning, but instead it fails on install.

I hope it helps people (I lost two hours figuring it out).

Read More

What's Expected of Us and others, by Ted Chiang

What's Expected of Us and Others is a collection of stories by Ted Chiang including

• Exhalation
• Dacey's Patent Automatic Nanny
• The Truth of Fact, the Truth of Feeling
• The Lifecycle of Software Objects
• What's Expected of Us
• The Merchant and the Alchemist's Gate
• The Great Silence

All the stories are speculative fiction, their subjects related to the meaning of life and evolution in relation to technology. His writing style is direct, unsophisticated, often based on first person perspective and dialogue. It's the ideas that he is exploring where he spends most of his efforts. All in all a nice collection of sci-fi, but not something extraordinary in nature.

Read More