Undo a Perforce changelist

I had this problem with Perforce where I accidentally Reconciled my offline work with all the files in /bin and /obj folders, resulting in a huge 6000+ file changelist. OK, simple one button mistake, surely there must be some one button undoing what I just did. It appears there is not.

In order to fix this I have to follow these steps:

1. Change the settings of Perforce to show files even in changelists larger than 1000 items (the default value)
2. Select by hand in the changelist window the files from obj and bin folders and using Revert on them
3. Revert the few other files that were unwanted in the changelist, like .suo and .user files - note that Revert on added files doesn't delete them, it just unadds them
4. Create a file with paths to ignore and then use p4 set P4IGNORE=<filename> for future reconcile work

What didn't work was adding a filename or path filter when visualizing the changelist, since that is a changelist filter, not a files filter. It will show you changelists that have files that contain the pattern, but not filter the files inside the changelists themselves.

For reference, the p4ignore file I used looked like this:

p4ignore
bin/Debug
obj/Debug
*.suo
*.user

Note that I also added the p4ignore file itself, although the file was not in any Perforce repository (yet).

"But, Siderite, you should use Git (or whatever source control is the newest fad at the moment)!" Wish that could, my friend, wish that I could.

Read More

The Trials (The Red #2) and Going Dark (The Red #3), by Linda Nagata

Having been so pleasantly surprised by First Light, the first book in The Red series, I quickly read the next two books: The Trials and Going Dark. However, possibly due to my high expectations, I have been disappointed by the continuation. Linda Nagata seemed to have reached that sweet spot between current tech trends and emergent future that makes stories feel both hard sci-fi and realistic. The integration between man and machine, the politics run by shadowy megarich "dragons" from the background, nuclear bombs detonated in major US cities, artificial intelligence and so on. The potential was immense!

Yet, the author chose to continue the story on the same flat note, like an ode to the Stockholm Syndrome, where the hero gets repeatedly coerced to run missions that at first seem bullshit, but in the end are rationalized as necessary and even dutiful by himself. The common intrusion of external forces into his emotional balance by way of direct brain stimulation also makes his feelings and motivations be completely isolated from the ones of the reader. A strange choice, considering the vast possibilities opened by the first book. Frankly, it felt like Nagata liked writing the first book and then was forced by publishers to make it "a trilogy", since that is the norm for fantasy and science fiction, but her heart wasn't really in it.

I don't want to spoil the ending, such as it is, but I will say it was disappointing as well, with no real closure for the reader of any of the important questions raised in First Light. Too bad, since I felt the story was beginning to touch on important subjects that needed to be discussed at a deeper level than just "boots on the ground".

Read More

Facebook relationship algorithms and how to use them to hack a relationship

I was listening to this Software Engineering Daily podcast about Facebook Relationship Algorithms and I had this weird idea. The more I was thinking about it, the more realistic it felt (as well as more than a bit creepy). Let me lay it out for you. The podcast is interesting in its own right, so go listen to it, it's instructive.

So they describe these metrics of your Facebook connections that they reached while trying to find an algorithm to detect your romantic relationship. They took a large sample of users that have declared their significant other and tried to find an automated way of predicting that from the other information Facebook had. The first idea was to use connectivity, one often used idea in sociology that the more common friends you have with someone, the closer you are, but it didn't quite work. One clear counterexample would be coworkers connected on social media. Instead, they realized that such functional connections often cluster, so you would have the cluster of coworkers, your family, your club friends, the people who share your hobby, etc. The romantic partner would be connected with many of these friends, but across clusters, in other words you would have many common friends that are not really connected with each other. By using this metric they called dispersion, they would guess with more than 50% accuracy from the first try who your partner is from the list of hundreds of your friends.

And here is my idea: why not reverse engineer it? Imagine you have someone you want to hook up with, but have no idea how to proceed. Maybe asking them on a date is not an option or maybe, like any engineer out there, you want to maximize the chances your experiment would work. Why not find the smallest subset of friends of that person that have the largest value of dispersion? So here is what this "hook me up" algorithm would do:

1. Collect the list of friends of your target
2. Find a sample that are well connected to the target, but less connected with each other
3. Approach each of them and befriend them

The result would be that you would become the "natural" choice for a relationship, by going backwards and reversing the direction of causality. Automatic stalking, courtesy of your friendly neighborhood software developer: Siderite! We live in an age in which information about us can be used or abused in innumerable ways and we become addicted and stuck to this way of relating. It's not a bad thing, but it has its drawbacks. It is good to know of them.

Read More

First Light (The Red #1), by Linda Nagata

First Light is Linda Nagata's first book in The Red series, which follows a military man landing right into the middle of an emergence event. Stuck between his duty as a soldier, his love for his girlfriend and father, the maniacal ambitions of an all powerful defense contractor queen and a mysterious God-like entity which seems to like him, our hero does what he can to survive and do good by his own principles.

At first I thought it was going to be one of those cheap soldiering books. It was short, written by a woman, and frankly I expected a standard pulp fiction "read it on a train" kind of thing. Instead I was blown away by the subtlety with which the characters are being explored and the way the story was constructed. I loved the book and I plan to read all the series. I started reading The Dread Hammer, which is another Nagata book, this time fantasy, but it doesn't even come close to First Light. I may even dislike it.

Anyway, I can't say much about the plot without spoiling it, but I can certainly recommend this book. As I said, it is short enough to read and see if it evokes the same feelings. Instead of hurting it, the female perspective of the author enhances the experience and makes it unique. The technical aspects are spot on and the writing style is fluid and easy to read. Top marks!

Read More

Switch any site to a dark/light scheme

Update October 2019: a CSS media feature (prefers-color-scheme) can be used in conjunction with this. A recent development, it's a media query that allows a browser to activate CSS code based on the theme set in the operating system. You set your preference in Windows or MacOS or wherever and then sites that use prefers-color-scheme will take advantage of that. Something like this:

@media (prefers-color-scheme: dark) {
  html,img, video, object, [style*=url] {
    -webkit-filter:invert(100%) !important;
    filter:invert(100%) !important;
  }

  body {
    background:black;
  }
}

I am a very light sensitive person. Shine a light in my eyes and you limit my productivity immensely. Not to mention it makes me irritable. Therefore I often have the desire to turn cheerful black on white sites to a dark theme, where the colors are reversed. I am sure other people have the same problem so I thought of building a browser extension to enable a switching button between the two.

The first problem is that I need to interrogate all the elements in a page, including the ones that will be created later. The second is that even so I would have problems determining the dominant color of images. But there is something I can use which makes all of this unnecessary: using the invert CSS filter! Since I already use a browser extension that injects my own styles in any site - it's called Stylish and I highly recommend it - all I have to do is apply a filter on the entire site, right?

Wrong! The problem is that when you invert an entire site, all images on the site get inverted, too. That also includes videos and Flash objects. The worst offenders here are the elements that sport a background image that is declared via CSS, since you can't create a CSS selector for them. I am going to present my partial solution and maybe you can help me find a more elegant or more complete one. Here is a general dark theme stylesheet, without the elements that have a background image declared via CSS (it does include those with a background image declared inline, though):

html,img, video, object, [style*=url] {
-webkit-filter:invert(100%) !important;
filter:invert(100%) !important;
}

body {
background:black;
}

What it does is invert the entire page (html), then reinvert video, img, and object elements, as well as those with "url" in the style attribute. In Chrome, at least, there seems to be a bug in the sense that the backgrounds of the direct child elements are not inverted, which means body, as the first child, needs to have the background set to black specifically. The hack to invert elements with "url" in style is pretty ugly, too.

What I think of a solution is this:

• inject Javascript to enumerate all elements present and future using document.createTreeWalker and Mutation Observers, check if they have a background image and if so, add a class to them
• inject the CSS above with an additional rule for the class for the elements with background image

However, this doesn't completely solve the problem. One of the major issues is the inverted colors sometimes look dumb. For example a red background turns cyan, white text on light gray background turns black test on dark gray background, which makes it hard to read. I've tried various other filters, like hue-rotate or contrast, but it doesn't really help. Detecting individual color patterns doesn't really work, as the filter attribute affects an element and all of its children. The CSS above only works because the images are inverted again when the entire page has been inverted.

The good news is that most of the time you may use the CSS above as a template, then add various rules (manually) to fix small issues with colors of backgrounds. Even if I don't package this in an extension, you have the power to create your own themes for various sites. Never again will you be subjects to the tyranny of the happy bright shiny people!

Update: if you have read this incredulously, thinking that I am preaching dark themes on a light themed blog, think again. If you go to Menu in the top-left corner you get the option to switch to a dark theme.

Read More

Eye openers from the SQLSaturday meeting

I was glad to attend the 565th SQLSaturday in Bucharest yesterday and, while all presentations were cool, I wanted to share with you some specific points that I found very revealing. Without further ado, here is the list:

• SQL execution plans are read from right to left - such a simple thing, but I remember when I was trying to read them from left to right and I didn't get anything. In SQL Server Management 2016 you also get a "live" version, which shows you an execution plan while it's executing. Really useful to see where the blocking operations are.
• Manually control your statistics update - execution plans are calculated based on statistics, but the condition for updating the statistics is to have changes in a number of 20% of the rows plus 500 of any table. This default setting is completely arbitrary and may cause a lot of pain. Not only updating the statistics blocks your table (which means more chances that the table will be locked when it is most used), but sometimes the statistics are not useful. One example are reports which may receive a startdate/enddate range or a count or something like that which makes the number of rows affected vary immensely with different parameters. Use OPTION(RECOMPILE) for that.
• Look for a difference between estimated and actual rows in a query plan, which leads to tempdb spills, which leads to unwanted IO operations - before a query, an execution plan is created or reused, based on statistics, as I was saying above. Once a plan has been chosen, though, it doesn't change during its execution. Basically what this means is that the structure of the plan remains unchanged between the estimated and actual plan. Also based on the plan, memory is requested and never changed. So if the plan asks for 10KB of memory and you need 1000KB, the rest of 990 will be stored and used from tempdb even if there is enough memory to put them in, since the memory requirements don't change from estimated to actual. The reverse is not much better, since a plan may ask for a lot of memory when it only needs little, thus making everything else on that machine have less available resources.
• SQL default settings suck - there was an entire presentation about that, it is useful to think a little about it. So many settings are legacy things that make no sense, like the initial database size, the autogrow size, index fill factors, maxdop (degree of parallelism), parallelism threshold, used memory (ironically, using all of it may be hurtful as it takes it away from other processes which leads to using the swap file), etc.
• Look for hard page faults - this counter is much useful than the soft page faults, which are fixable faults. A hard page fault is indicative of unnecessary IO operations, which are orders of magnitude slower than memory use.

There are a lot more things that I want to explore now, since I participated to the event. You may find the files for the presentations in the same place as the full list of talks at SQLSaturday.

Read More

The Dragonbone Chair, by Tad Williams

Tad Williams probably fancies himself as another Tolkien: he writes long decriptions of lands and people and languages, shows us poetry and songs, tells us about the rich history of the land. And all of this while we follow yet another common, but good boy, with a mysterious ancestry, while he and his merry band of helpers fight THE DARK ONE. It's the same old story, with the hapless youth that is guided by wise but not forthcoming people who tend to die, leave or otherwise shut up before the hero gets the whole story and can do anything about it. Regardless, he is young and lucky, so it's OK.

If The Dragonbone Chair would have been fun or interesting or at least show us a character that we could care about, this book would have been readable. As such it's a trope filled, boring and sleep inducing thing. You have to wait until half of the book to see the things that you predicted would happen from the first few chapters. I couldn't even finish it. More than two thirds in the book and there is no significant part of the story that involves either dragons or chairs.

Bottom line: it sucks!

Read More