Detecting the language of a text in C#

Short post about how to use language detection capabilities in your application. I will be demonstrating NTextCat, which is a .NET port of text_cat, a Perl script, itself an implementation of a whitepaper published in 1994 called N-Gram-Based Text Categorization.

There are four steps to language detection using NTextCat:
• Reference NTextCat - the library is now available as a NuGet package as well
• Instantiate an identifier factory (usually RankedLanguageIdentifierFactory)
• Get an instance of a RankedLanguageIdentifier from the factory by loading a language XML file
• Call the Identify method on your text and get a list of languages in the order of the probability that the text in that language

Here is a piece of code that does that using the core XML file published with the library. Remember to add the XML to your project and set its property of Copy to Output Directory.

public class LanguageProcessor
{
    private RankedLanguageIdentifier _identifier;

    public string IdentifyLanguage(string text)
    {
        if (_identifier == null)
        {
            var file = new FileInfo(Path.Combine(AppDomain.CurrentDomain.BaseDirectory, "LanguageModels/Core14.profile.xml"));
            if (!file.Exists)
            {
                throw new FileNotFoundException("Could not find LanguageModels/Core14.profile.xml to detect the language");
            }
            using (var readStream = File.OpenRead(file.FullName))
            {
                var factory = new RankedLanguageIdentifierFactory();
                _identifier = factory.Load(readStream);
            }
        }
        var languages = _identifier.Identify(text);
        var mostCertainLanguage = languages.FirstOrDefault();
        if (mostCertainLanguage != null)
        {
            return mostCertainLanguage.Item1.Iso639_3;
        }
        return null;
    }

}

There are a lot of XML files, some taken from Wikipedia, for example, and handling 280+ languages, but for the casual purpose of finding non-English text in a list, the core one will suffice.

Read More

A log4net custom appender that creates JIRA issues and notifies users

It has been a while since I've blogged something technical. It's just that nothing I did seemed to be worthy of this majestic blog... Well, jokes aside, here is a post detailing my log4net JIRA appender.

log4net is a popular (if not the most popular) logging framework for .NET out there. Its strength lies in its configurability, the possibility to create custom loggers, custom appenders, custom filters, etc. I will be talking about a custom appender, a class that can be loaded by log4net to consume the logged lines and put them somewhere. For example to make an application that uses log4net to write the log to the console all you do is configure it to use the console appender. The JIRA appender takes the log output and creates issues in JIRA, notifying users afterwards. JIRA is a tracker for team planning. It is also very popular.

In order to create an appender, one references the log4net assembly (or NuGet package) and then creates a class that inherits from AppenderSkeleton. We could implement IAppender, but the skeleton class has most of what people want from an appender. The next step is to override the Append method and we are done. We don't want to create an issue with each logged line, though, so we will make it so that it creates the issue after a period of inactivity or when the logger closes. For that we use the CancellationTokenSource class to create delayed actions that we can cancel and recreate. We also need to override OnClose().

For the JIRA Api I used a project called AnotherJiraRestClient, but I guess one can used anything out there. You will see that the notify functionality is not implemented so we have to add it.

Here is the appender source code:

using AnotherJiraRestClient;
using log4net.Appender;
using System;
using System.Collections.Generic;
using System.IO;
using System.Linq;
using System.Net;
using System.Text;
using System.Threading;
using System.Threading.Tasks;

namespace JiraLog4netAppender
{
    public class JiraAppender : AppenderSkeleton // most appender functionality is found in this abstract class
    {
        private List<string> _notifyUsers;

        // properties of the appender, configurable in the log4net.config file as children elements of the appender
        public string url { get; set; } // the url of the JIRA site
        public string user { get; set; } // the JIRA user
        public string password { get; set; } // the JIRA password
        public string project { get; set; } // the JIRA project
        public string notify // a comma separated list of JIRA users who will be notified of the issue's creation
        {
            get
            {
                return string.Join(", ", _notifyUsers);
            }
            set
            {
                _notifyUsers.Clear();
                if (!string.IsNullOrWhiteSpace(value))
                {
                    _notifyUsers.AddRange(value.Split(',').Select(s => s.Trim()).Where(s => !string.IsNullOrWhiteSpace(s)));
                }
            }
        }

        CancellationTokenSource _ts;
        StringWriter _sw;
        Task _task;
        private JiraClient _jc;
        private string _priorityId;
        private string _issueTypeId;

        private object _writerLock=new object();

        public JiraAppender()
        {
            _notifyUsers = new List<string>();
            _ts = new CancellationTokenSource(); // use this to cancel the Delay task
            _sw = new StringWriter();
        }

        protected override void Append(log4net.Core.LoggingEvent loggingEvent)
        {
            this.Layout.Format(_sw, loggingEvent); // use the appender layout to format the log lines (warning: for this you need to override RequiresLayout and return true)
            _ts.Cancel(); // cancel the task and create a new one. This means as long as the logger writes, the 5 second delay will be reset
            _ts = new CancellationTokenSource();
            _task = Task.Delay(5000, _ts.Token).ContinueWith(writeToJira, _ts.Token); // after 5 seconds (I guess you could make this configurable as well) create a JIRA issue
        }

        protected override bool RequiresLayout
        {
            get
            {
                return true;
            }
        }

        /// <summary>
        /// write to jira, either when 5 seconds of inactivity passed or the appender is closed.
        /// </summary>
        /// <param name="task"></param>
        private void writeToJira(Task task)
        {
            string s;
            lock (_writerLock) // maybe the method was already in progress when another one was called. We need to clear the StringWriter before we allow access to it again
            {
                s = _sw.ToString();
                var sb = _sw.GetStringBuilder();
                sb.Clear();
            }
            if (!string.IsNullOrWhiteSpace(s))
            {
                writeTextToJira(s);
            }
        }

        private void writeTextToJira(string text)
        {
            ensureClientAndValues();
            var summary = "Log: " + this.Name; // the issue summary
            var labels = new List<string> // some labels
            {
                this.Name, this.GetType().Name
            };
            var issue = new AnotherJiraRestClient.JiraModel.CreateIssue(project, summary, text, _issueTypeId, _priorityId, labels); // create the issue with type Issue and priority Trivial
            var basicIssue = _jc.CreateIssue(issue);
            _jc.Notify(basicIssue, _notifyUsers, "JiraAppender created an issue", null, null); // notify users of the issue's creation
        }

        /// <summary>
        /// Make sure we have a JiraClient and that we know the ids of the Issue type and the Trivial priority
        /// </summary>
        private void ensureClientAndValues()
        {
            if (_jc == null)
            {
                _jc = new JiraClient(new JiraAccount
                {
                    ServerUrl = url,
                    User = user,
                    Password = password
                });
            }
            if (_priorityId==null) {
                var priority = _jc.GetPriorities().FirstOrDefault(p => p.name == "Trivial");
                if (priority == null)
                {
                    throw new Exception("A priority with the name 'Trivial' was not found");
                }
                _priorityId = priority.id;
            }
            if (_issueTypeId == null)
            {
                var meta = _jc.GetProjectMeta(project);
                var issue = meta.issuetypes.FirstOrDefault(i => i.name == "Issue");
                if (issue == null)
                {
                    throw new Exception("An issue type with the name 'Issue' was not found");
                }
                _issueTypeId = issue.id;
            }
        }

        protected override void OnClose() //clean what you can and write to jira if there is anything to write
        {
            _ts.Cancel();
            writeToJira(null);
            _sw.Dispose();
            _ts.Dispose();
            _task = null;
            base.OnClose();
        }

    }
}

As I said, AnotherJiraRestClient does not implement the notify API call needed to inform the users of the creation of an issue, so we need to change the project a little bit. Perhaps when you are implementing this, you will find notify already there, with a different format, but just in case you don't:

• add to JiraClient the following method:

  public void Notify(BasicIssue issue, IEnumerable<string> userList, string subject, string textBody, string htmlBody)
  {
      var request = new RestRequest()
      {
          Method = Method.POST,
          Resource = ResourceUrls.Notify(issue.id),
          RequestFormat = DataFormat.Json
      };
      request.AddBody(new NotifyData
      {
          subject = subject,
          textBody = textBody,
          htmlBody = htmlBody,
          to = new NotifyTo
          {
              users = userList.Select(u => new User
              {
                  active = false,
                  name = u
              }).ToList()
          }
      });
      var response = client.Execute(request);
      if (response.StatusCode != HttpStatusCode.NoContent)
      {
          throw new JiraApiException("Failed to notify users from issue with id=" + issue.id+"("+response.StatusCode+")");
      }
  }

• add to ResourceUrls the following method:

  public static string Notify(string issueId)
  {
      return Url(string.Format("issue/{0}/notify", issueId));
  }
  

• create the following classes in the JiraModel folder and namespace:

  public class NotifyData
  {
      public string subject { get; set; }
      public string textBody { get; set; }
      public string htmlBody { get; set; }
      public NotifyTo to { get; set; }
  }
  
  public class NotifyTo
  {
      public List<User> users { get; set; }
  }
  
  public class User
  {
      public string name { get; set; }
      public bool active { get; set; }
  }

Here is an example configuration. Have fun!

<log4net>
   <appender name="JiraAppender" type="JiraLog4netAppender.JiraAppender, JiraLog4netAppender">
      <url value="https://my.server.url/jira"/>
      <user value="jirauser"/>
      <password value="jirapassword"/>
      <project value="jiraproject"/>
      <notify value="siderite,someotheruser" />
      <layout type="log4net.Layout.PatternLayout">
          <conversionPattern value="%date [%thread] %-5level %logger [%property{NDC}] - %message%newline" />
      </layout>
        <filter type="log4net.Filter.LevelRangeFilter">
          <levelMin value="WARN" />
          <levelMax value="FATAL" />
        </filter>
   </appender>
     <root>
     <level value="DEBUG" />
         <appender-ref ref="JiraAppender" />
   </root>
</log4net>

Read More

Algorithms as cash, now algorithms in company board of directors

The world is getting into a trend of using software and hardware in places traditionally reserved for humans or "real life" applications. An extraordinary example of this, something that I consider way ahead of its time, is the creation of Bitcoins. You see, money is supposed to be an abstraction of wealth, but wealth, for all intents and purposes, is determined by some material such as gold or gems. It was the previous layer of abstraction, right after people would do commerce using sheep and goats and women: they used something they mined from under the earth to equate value of real things. Enter Bitcoin, something that you "mine" using processing power, using an algorithm, having weight of value determined only by the computation and strength of encryption used. After all, gold in itself had no value either when it was used as currency: it only had weight of value by the effort of getting out of the ground and its rarity.

I didn't write about Bitcoins until now because frankly I don't get how it works. I get the gist of it, but I would like to understand the algorithm behind, which relies heavily on encryption. I am terrible at any type of cryptography, though. However, the story that prompted me to write a blog entry today is, in a way, even weirder than Bitcoins. A BBC article entitled "Algorithm appointed board director" describes that venture capital firm Vital assigned an automated algorithm a vote in the decision to invest or not in a company. Of course, this has been done before, but indirectly. Some guy would use a computer in some basement of the office building, trying to determine if a company is viable or not using computer programs. He would then pass the information to some manager, who would pass it on to his manager until it got to some member of the board who voted in the council. This story, though, describes a complete removal of middlemen. Of course, there is still some techie that takes care of the IT department. He probably could sway the algorithm to vote for a company he personally likes, if he wanted too, but the techies are always subsumed in the concept of "machine", perhaps until they learn to service each other. The article says the same thing, really, but fails to recognize the importance of the removal of that chain of middle manager who's jobs are only to filter information from downstairs and decisions from upstairs. It is akin to Blake's 7, where the seventh was a machine.

I started this blog entry talking about a trend and I've just described two applications, but there are really a lot more. A trend is also powered by public perception, which is powered by media outlets. So I am not only talking about the self driving cars, the killing drones, the self landing probes, the algorithmic news aggregators and even writers, but also about the TV series and movies that are spreading the seed of this idea of cyber-revolution: Intelligence, Transcendence, The Machine, Robocop, even Silicon Valley, if you've seen the episode where the self driving car abducts someone. All of these are released this year, as well.

Of course, there has always been some sort of golem like creature in human fantasy, but it was always about creatures/machines trying to gain unrestricted power, power that belongs to humans only or maybe only to gods (in essence, something that should not be had). Some of them, like Star Trek's Data, were benign, always trying to achieve the "humanity" evident in his colleagues, but even in that story there was always the underlying idea that if he managed to reach his goal, then Data would become an immortal creature of human ability, but also a lot more than that: a superhuman. In this decade we see the rise of transhumanism, something I wholly support, the self-evolution of the human being, but also of the singularity, the self-evolution of machines to the point where we are left behind. This very familiar notion of competition between memes makes it accessible, in that "us or them" kind of way that every human, no matter how idiotic, resonates with, but also interesting because of the commonality found in the two concepts: it's either the machines overtaking the evolution rate of the human animal or the humans accelerating the evolution of the human animal. It's all about overcoming that beast that exists in us and that we think of as "not me", but that guides most of our existence. I hope no amount of adolescent fantasies and "emotion over matter" garbage will be able to undo this. And I am terribly excited because I believe that by the end of my life I will see (if not become) one or the other happening.

Read More

Adventures in the .Net file system library System.IO

Intro (click to hide)

Sometimes, after making some software or another and feeling all proud that everything works, I get an annoying exception that the path of the files my program uses is too long, the horrible PathTooLongException. At first I thought it was a filesystem thing or an operating system thing, or even a .Net version thing, but it is not. The exception would appear in .Net 4.5 apps running on Windows 7 systems that used NTFS on modern hardware.

Lazy as any software dev, I googled for it and found a quick and dirty replacement for System.IO called Delimon that mitigated the issue. This library is trying to do most of what System.IO does, but using the so called extended-length paths, stuff that looks like \\?\C:\Windows, with external functions accessed directly from kernel32.dll. All good and nice, but the library is hardly perfect. It has bugs, it doesn't implement all of the System.IO functionality and feels wrong. Why would Microsoft, the great and mighty, allow such a ridiculous problem to persist in their core filesystem library?

And the answer is: because it is a core library. Probably they would have to make an enormous testing effort to change anything there. It is something from a managed code developer nightmare: unsafe access to system libraries, code that spans decades of work and a maze of internal fields, attributes, methods, properties that can be used only from inside the library itself. Not to mention all those people who decided to solve problems in core classes using reflection and stuff. My guess is that they probably want to replace file system usage with another API, like Windows.Storage, which, alas, are only used for Windows Phone.

In this blog post I will discuss the System.IO problems that relate to the total length of a path, what causes them and possible solutions (if any).


Let's start with the exception itself: PathTooLongException. Looking for usages of the exception in the mscorlib.dll assembly of .Net 4.0 we see some interesting things. First of all, there is a direct translation from Windows IO error code 206 to this exception, so that means that, in principle, there should be no managed code throwing this exception at all. The operating system should complain if there is a path length issue. But that is not the case in System.IO.

Most of the other usages of the exception come from the class PathHelper, a helper class used by the System.IO.Path class in a single method: NormalizePath. Wonderful method, that: internal static unsafe. PathHelper is like a multiple personality class, the active one being determined by the field useStackAlloc. If set to true, then it uses memory and speed optimized code, but assumes that the longest path will always be 260. That's a constant, it is not something read from the operating system. If set to false, the max path length is also provided as a parameter. Obviously, useStackAlloc is set to true in most situations. We will talk about NormalizePath a bit later.

The other usages of the PathTooLongException class come from two Directory classes: Directory and LongPathDirectory. If you instantly thought "Oh, God, I didn't know there was a LongPathDirectory class! We can use that and all problems disappear!", I have bad news for you. LongPathDirectory is an internal class. Other than that it seems to be a copy paste clone of Directory that uses Path.LongMaxPath instead of hardcoded constants (248 maximum directory name length, for example) or... Path.MaxPath. If you thought MaxPath and LongMaxPath are properties that can be set to fix long path problems, I have bad news for you: they are internal constants set to 260 and 32000, respectively. Who uses this LongPathDirectory class, though? The System.IO.IsolatedStorage namespace. We'll get back to this in a moment.

Back to Path.NormalizePath. It is a nightmare method that uses a lot of internal constants, system calls, convoluted code; it seems like someone deliberately tried to obfuscate its code. It's an internal method, of course, which makes no sense, as the functionality of path normalization would be useful in a lot of scenarios. Its first parameter is path, then fullCheck, which when true leads to extra character validation. The fourth parameter is expandShortPaths which calls the GetLongPathName function of kernel32.dll. The third parameter is more interesting, it specifies the maximum path length which is sent to PathHelper or makes local checks on the path length. But who uses this parameter?

And now we find a familiar pattern: there is a class (internal of course) called LongPath, which seems to be a clone of Path, only designed to work with long paths. Who uses LongPath? LongPathDirectory, LongPathFile and classes in the System.IO.IsolatedStorage namespace!


So, another idea becomes apparent. Can we use System.IO.IsolatedStorage to have a nice access to the file system? No we can't. For at least two reasons. First of all, the isolated storage paradigm is different from what we want to achieve, it doesn't access the raw file system, instead it isolates files in containers that are accessible to that machine, domain, application or user only. Second, even if we could get an "isolated" store that would represent the file system - which we can't, we would still have to contend with the string based way in which IsolatedStorage works. It is interesting to note that IsolatedStorage is pretty much deprecated by the Windows 8 Windows.Storage API, so forget about it. Yeah, so we have LongPathDirectory, LongPathFile and LongPath classes, but we can't really use them. Besides, what we want is something more akin to DirectoryInfo and FileInfo, which have no LongPath versions.

What can we do about it, then? One solution is to use Delimon. It has some bugs, but they can be avoided or fixed, either by the developers or by getting the source/decompiling the library and fixing the bugs yourself. A limited, but functional solution.
An interesting alternative is to use libraries the BCL team published for long path access: LongPath which seems to contain classes similar to the ones we find in mscorlib, but it's latest release is from 2010 or Zeta long paths which has a more recent release, 2013, but is completely different, using the FileInfo and DirectoryInfo paradigm, too.
Of course, you can always make your own API.

Another solution is to be aware of the places where the length limitation appears and avoid them via other type of development, in other words, a file system best practices compilation that eventually turns into a new file system API.

Both solutions coalesce into using some other library instead of System.IO. That's horrible and I think a stain on .Net's honor!


So let's see where the exception is actually thrown.

I've made some tests. First of all, I used FAR Manager, a file manager, to create folders of various lengths. The longest one was 255, before I got an exception. To my surprise, Windows Explorer could see it, but it could not open or copy/rename/delete it. I reduced the size of its name until the total size of the path was 260, then I could manipulate it in Windows Explorer. So there are external reasons for not creating paths as long, but we see that there are tools that can access files like that. Let's attempt to create some programatically.

System.IO.Directory.CreateDirectory immediately fires the exception. DirectoryInfo has no problem instantiating with the long path as the parameter, but the Create method throws the same exception. Any attempt to create a folder of more than 248 characters, even if the total path was less than 260 characters, failed as well.

However, with reflection to the rescue, I could create paths as long as 32000 characters and folders with names as long as 255 characters using our friend LongPathDirectory:

var longPathDirectoryType = typeof(System.IO.Directory).Assembly.GetTypes().First(t=>t.Name=="LongPathDirectory");
var createDirectoryMethodInfo = longPathDirectoryType.GetMethod("CreateDirectory", System.Reflection.BindingFlags.Static | System.Reflection.BindingFlags.NonPublic);
createDirectoryMethodInfo.Invoke(null, new object[] { path });

What about files? FAR Manager threw the same errors if I tried to create a filename larger than 255 characters. Let's try to create the same programmatically.

File.Create threw the exception, as well as FileInfo.Create and the FileStream constructors.

So can we use the same method and use LongPathFile? No! Because LongPathFile doesn't have the creating and opening functionalities of File. Instead, FileStream has a constructor that specifies useLongPath. It is internal, of course, and used only by IsolatedStorageFileStream!

Code to create a file:

var fileStreamConstructorInfo = typeof(System.IO.FileStream).GetConstructor(System.Reflection.BindingFlags.NonPublic|System.Reflection.BindingFlags.Instance,null,
        new Type[] { 
        typeof(string) /*path*/, typeof(System.IO.FileMode) /*mode*/, typeof(System.IO.FileAccess) /*access*/, 
        typeof(System.IO.FileShare) /*share*/, typeof(int) /*bufferSize*/, typeof(System.IO.FileOptions) /*options*/, 
        typeof(string) /*msgPath*/, typeof(bool) /*bFromProxy*/, typeof(bool) /*useLongPath*/, typeof(bool) /*checkHost*/},null);
    var stream = (System.IO.Stream)fileStreamConstructorInfo.Invoke(new object[] {
        path, System.IO.FileMode.Create, System.IO.FileAccess.Write,
        System.IO.FileShare.None, 4096, System.IO.FileOptions.None,
        System.IO.Path.GetFileName(path), false, true, false
    });

Horrible, but it works. Again, no filenames bigger than 255 and the exception coming from the file system, as it should. Some info about the parameters: msgPath is the name of the file opened by the stream, if bFromProxy is true the stream doesn't try to demand some security permissions, checkHost... does nothing :) Probably someone wanted to add a piece of code there, but forgot about it.

Why did I use 4096 as the buffer size? Because that is the default value used by .Net when not specifying the value. Kind of low, right?

Now, this is some sort of midway alternative to using a third party file system library: you invoke through reflection code that is done by Microsoft and hidden for no good reason. I don't condone using it, unless you really need it. What I expect from the .Net framework is that it takes care of all this for me. It seems (as detailed in this blog post), that efforts are indeed made. A little late, I'd say, but still.

Read More

Wild Cards, an alternate fiction world edited by George R. R. Martin

I accidentally heard of the Wild Cards books a few weeks ago, but the concept fascinated me. The plot is that of an alternate America in which an alien virus caused massive deaths, but also strange mutations in 1946. The virus, something a bunch of aliens wanted to test on Earth as a bioweapon, kills 90% of its victims, mutates in horrible forms 9% of them, but also gives powerful abilities to but 1%. This 1% are called Aces, while the deformed ones are called Jokers, the analogy with a deck of cards giving the series its name. What is even more interesting is that this is like an open source literary universe, edited by George R. R. Martin, but in which a lot of writers are creating content. First book was published in 1987, and more and more were published and still are in the present. Some of them are collections of stories, some of them are full featured books; they all happen in the same universe, same heroes, and Martin is making sure they are ordered chronologically and have consistency. I found the concept intriguing.

Anyway, I've already read the first two books and started reading the third and I like it. It has the feeling of the Union Dues stories and Watchmen: dark, bleak sometimes, pulling no punches when it is about human pettiness, base desires or social ugliness. It also has some positive messages and classic "good wins in the end" stories. I was impressed by the faithful following of American history, including savage McCarthy witch hunts against jokers and aces, a Vietnam war with the appropriate Flower Power anticulture, complete with actual historical figures that somehow get affected by the virus (like the werewolf Mick Jagger).

Now I am not saying that this is the best book series ever written. It certainly has boring or lagging parts, some of it is slightly puerile (after all it is a superhero series), but so far I enjoy it. It is worth mentioning that there are 12 books published by Bantam Books before a "new cycle" appears, published by Baen, then two from ibooks - a publishing house that suddenly went down - and now Tor Books is apparently publishing the rest (a revival, it is called). 21 books so far and another one set to be released this year. In other words, some books may be better than others and I can only discuss my feelings after reading the first two.

In conclusion, it felt weird to not have heard of these books until now. Certainly they gained more popularity with Game of Thrones getting all this attention, but still, an alternate history superhero series of more than 20 books should have had some impact on me so far. I am glad I finally got wind of them and I enjoy them so far. I hope I find a system of filtering the books, though. I don't know if I am ready to read 20 books at once.

Read More

A Fire Upon the Deep, by Vernon Vinge

A Fire Upon the Deep is a really strange book. The writing style of the author, Vernon Vinge, reminds me a lot of Asimov: the describing of scenes that are clear in his head, but the willingness to move immediately over concepts or ideas that are not, the sometimes obnoxiously long dialogues, the direct way of saying it as it is. In fact, without knowing anything about Vinge, I bet that he is an engineer, maybe even a computer scientist. And I won. The concepts of the book range from high hard sci-fi to naive depictions of kilobyte per second communication. So, in all earnest, I thought the book was amateurish, meanwhile reading it from cover to cover in a few days, just like one of Asimov's books. Still, for a book written in 1992, it felt terribly outdated.

The plot is a combination of story arches. The main one is the emergence of an evil artificial intelligence that plans to take over the galaxy and the quest to retrieve the ultimate weapon that would defeat it. Then there is the story of a medieval alien race of rat-wolf analogues that think in packs, making an individual from several bodies acting together. And finally there is the internal dynamic of the group that embarks in this quest of quests. Some interesting ideas are being thrown around, but almost always with terrible naivete, such as the Internet Relay Chat type of communications between interstellar civilisations or the distinction of several Zones of the galaxy in which technology and space travel can work at various speeds. The alien creatures, in their vast majority, are badly described with embarrassing slip-ups like using the word "zombie" or some other typical human colloquialisms in an alien context, however some ideas are ingenious. I will list here the way the "tines" use ultrasonics to group think and act like singular entities, while being able to use sound for "interpack" communication. The way a soul of such a creature is affected by the death, addition, injury or indeed torture of one of the individual bodies is also explored, with various degrees of success. The creation or manipulation of an entire race of people in order to further the goals of a "godly" intelligence is also an interesting twist.

To sum it all up, from the three main story arches, the pack intelligence aliens one was the most thorough, while the one relating to AI and space travel and communication was the least. Amazing coming from a computer scientist. In fact, I would have liked the book more if its sole subject was about the accidental marooning of two children on a starship in the middle of a strange alien feudal world. The rest felt clunky and frankly completely ridiculous in most cases. I still read it with interest, although I don't intend to read anything else from Vinge.

Read More

TV Series I've Been Watching, part 18

Time for another TV series watch list. Let's see what has changed in the last four months!

• The Good Wife - This is a show that I watch with my wife, so I had to wait for her to watch it with me, but, after a short break, The Good Wife is back again. Pardon the pun, couldn't help myself. Towards the end of the season we see a very important character die off. I don't want to spoil it, but it is interesting to see how it will continue from here.
• The Walking Dead - The new formula of the show is that, after running from the other group of humans and a lot of zombies, the group has split. So each episode we see what stupid people do when they are all alone. Well, stupid things, of course. They did go for the shock value with one of the characters going all psychotic and killing another, but it is still stupid. How can they maintain the idea of the zombie threat, when they are just slow, moaning corpses, that walk in small groups? What happened with the migrating hordes? The end of the season finds everyone reunited, but facing another threat.
• Arrow - Manu Bennet's character comes to the foreground, as well as a lot of others, that are less interesting. I really like Manu Bennet, but his character lacks consistency.
• Elementary - After defeating Moriarty and making a complete ass out of inspector Lestrade and after they showed the human side of Sherlock, it's kind of hard to distinguish Elementary from all the other police procedurals out there. They are trying to let Lucy Liu's character get more of the spotlight, which goes to further erode the mythos that makes Sherlock... well, Sherlock! A mysterious story arch regarding his brother seems to be looming, too, which probably means that when we will see Sherlock's dad appear, we will know the show is close to an end.
• The Tomorrow People (2013) - after a direct confrontation between the main character and his evil uncle, right after finding out his mother also has superpowers, we get to see The Founder in a different light, only for him to suddenly assert control and play the nice guy. A lot of melodrama in this show about people with superpowers who can't seem to be able to leave their country and live happily in a tropical island somewhere.
• The Originals - Out of boredom I continue to watch this on fast forward. Petty feudal schemes and plots, brotherly love and infighting, a lot of posturing. Each episode has about 5 minutes of interesting material, if you discount the beautiful girls, and that is what I am watching.
• Marvel's agents of S.H.I.E.L.D. - They finally find the secret of agent Coulson's survival and start a new arch of searching for the origin of the alien corpse with magical blood. Meanwhile, they save Sky by using the same procedure. I can't wait for her to turn blue or something. The Hydra enters the equation and we find the identity of The Clairvoyant, while a member of the team is suddenly revealed to be a sleeper agent.
• Killer Women - Last blog entry I wrote about this suggested the show would quickly be cancelled, because Hollywood doesn't know how to make non pornographic shows around female characters. I also think the public (especially in the US) doesn't really like women in positions of power in their fantasies. I was right. The show is gone.
• Ripper Street - The show might still have a chance. According to news, the series might be continued on Amazon.
• Sherlock - I don't really know what this show is about anymore. Sherlock is unhinged, probably psychotic, his friend has some real issues being friends with this guy and Sherlock's brother is probably the worse of them all, including Moriarty. Who, BTW, didn't die either.
• Babylon - A new British series, codirected and produced by Danny Boyle, about a woman PR person trying to direct and improve communication from the police department to the public. The premise is not that great, but I really like the main actress, Brit Marling, and the show could become a lot more.
• Banshee - A new season with a lot of people dying. Lots! The show continues to keep one viscerally on the edge, even if what happens doesn't make a lot of sense. It's like the Smells Like Teen Spirit of TV shows. I love it, I just don't know why. Funny enough, my father, the man who is always making fun of my choice of entertainment, also likes it!
• Bitten - The incredibly beautiful Laura Vandervoort is a werewolf. Unfortunately, she is not the only one. She is part of a pack of pompous asses who are being harassed by a pack of psychopath werewolves. I mean that literally: someone turns psychopaths into werewolves in order to exact their revenge on said pack. The series is incredibly boring. Not even Laura's naked shots can't save this show. Wooden acting, bad premise, inconsistencies at every step. Ugh!
• Black Sails - It's pirate season! A show about pirates that has some very interesting characters and some cool ideas, not with cliched drunkards spouting "Arghh" every sentence. Unfortunately it really drags on. It might pick up the pace soon, though. Without the need for additional artificial drama, hopefully.
• Dilbert - I watched the Dilbert animated cartoons until I couldn't anymore. Some episodes are really funny, but I don't think the show makes justice to the comic strip. But if you are an engineer in some corporation, it should be very therapeutic :)
• Fat Tony and Co - An Australian drama about a real drug lord that the police tried to catch. It is kind of boring, though. The guy is just a businessman who happens to deal in drugs. Other than that he is a normal bloke. His family and mates are the same. The police people are normal people, too. It kind of makes them all alike and their conflict meaningless.
• Flemming, the Man Who Would Be Bond - A four episode British miniseries about the life of Ian Flemming, the writer of the James Bond books. Charismatic characters and a good show. You should see it, especially if you liked the James Bond books and/or movies.
• From Dusk Till Dawn - Yes! The series version of the film with the same name has finally come to life. A vampiric cult of people believing in strange gods, bank robbers on the run, a disillusioned priest on a journey of rediscovery with his annoying adolescent kids, an angry ranger set on revenge, there are a lot of interesting characters. The story, though, kind of drags on, with no relief in sight. This might have worked for a movie, but for a series, you have to give something in every episode, you can't just finalize every story arch at the end of a season
• Helix - Oh, man! How can you make a show about viral outbreaks in an Antarctic station and mess it up so completely?! To borrow a page from the Romanian comic Pidjin, I think SyFy are hard at work making science-fiction fiction. I can't think of anything good about this show, except the Japanese actor, Hiroyuki Sanada, who is always good, but who I think made a horrible career move agreeing to play in this shit.
• Hinterland - Welsh police procedural. Dark, centered around small communities and their secrets. A kind of desolate cross between Midsommer Murders and Broadchurch. I like it.
• House of Cards - Season 2 is out. Haven't watched it yet, as this is also a show that I watch with my wife, but I can't imagine Kevin Spacey messing up.
• Intelligence - Another US government agency is saving the world. This time is Cyber Command. They implanted a chip in the head of the only guy from Lost who wasn't too annoying and now he can hack things instantly, connect to databases, etc., while also keeping his Special Forces skills and macho charm. I like the show, even if it is kind of stupid, mostly because of the charisma of the main actor, Josh Holloway, and that of John Billingsley, who plays the scientist. The condescending bitch that is the partner and protector of the headchip guy, a good looking girl who acts like a log, is subtracting points out of it all, no matter how tight her pants are.
• Looking - A show about gay men who was touted as being not stereotypical and actually showing real gay life. I watched the first episode and, if that is real gay life then it is really... errr.. gay! As in boring. Imagine trying to be interested in Sex in the City starring hairy men. Ugh! Also, it looked really filled with gay clichees to me. But maybe I just don't know what I am talking about. I didn't like the show at all.
• Ressurection - Remember when I recommended you a nice French series about dead people suddenly appearing live and well in their hometown? The Americans cloned Les Revenants into their own version. The script is not exactly the same, though, it's a real adaptation. All I can say is that what made me feel curious and connected to the characters in Les Revenants is missing in Ressurection. And why did they have to center it all around an FBI agent? Not enough cop shows? It may be too early to tell if the show is going to get better or not, so I will keep the watch.
• Star Crossed - Oooh, another sci-fi show! This one is about an alien race that crashlands on Earth and we decide to keep them all in a concentration camp while haters roam free and try to kill them. Then an integration program is born, where Atrian teenagers are accepted into a human high school. Then it turns into a sort of Twilight meets Defiance via the Black liberation movement. It's not horrible, but it is certainly bad.
• The 100 - Yet another sci-fi show about adolescents. But this one started nice enough. People live in a giant space station because they nuked the Earth. The series starts with 100 teenager death row convicts sent to the planet to ascertain if it is survivable. Lord of the Flies meets Elysium, maybe? The first episodes are intriguing, even if they already showed crass leaps in logic. I hope this one will be good, although I remember hoping the same about Lost and look how it turned out.
• The After - And another sci-fi. Or so I thought. Something bad happens and some people get trapped in a building parking. They manage to escape and reach the house of one of them. Then it gets freaky when they realize they are all born on the seven of March and meet a demon like creature with a tatooed body that encompasses the individual tattoos of all of the people in the group. Also the prostitute said something about the book of Revelations, so I think it is actually a bad religious apocalypse show. Let's see where it goes.
• The Assets - I first thought it was an alternative to The Americans, a show about the CIA agent Aldrich Ames, who willingly became a double agent for the KGB. The show was nice, no special effects, no artificial drama, but from the beginning it didn't seem like the TV network wanted the show to begin with. After just two episodes they cancelled it. I can't say I loved it, but it was certainly better than a lot of crap that keeps sticking to the TV screens these days.
• The Americans - The second season had begun. I like the show a lot. Both main characters are fantastic in their roles and make it all believable. Their annoying daughter started to suspect things and someone just killed some undercover KGB agents and no one knows who. Keri Russell does a very good job jumping from Felicity to KGB agent, while Matthew Rhys comes out very well as a chameleon capable of impersonating just about anyone.
• The Musketeers - You can't have pirates and not have musketeers. Loosely based on the Dumas books, this is a show about D'Artagnan being the coolest guy that ever lived and everybody taking an interest in him. Peter Capaldi is cardinal Richelieu. Strange choice, considering he is also Doctor Who, but very befitting the role. The show is not bad, but it feels so incredibly fake. I hope it will get better soon.
• The Red Road - Jason Momoa plays in this as a bad ass native American who blackmails a cop in order to maintain his drug running operation. He plays well the role, looking all angry and violent and remorseless, but the movie subject is terribly boring. It feels like the type of show that gets cancelled and I personally have decided not to watch it anymore.
• True Detective - An HBO project that features Woody Harrelson and Matthew McConaughey as cops in a God forsaken region of the US where religious fanatics and child rapists (sometimes the same person) roam free. Complex characters, eight episode seasons, very good acting and an interesting story make this one of the best shows around. It's not all rosy, though. In the first season more than three quarters was suspense and exposition. You always feel something will be going on, but it takes entire episodes before something does. It will be interesting to see if they continue with other seasons and, if they do, will they retain the same actors. The story arch has pretty much ended with the last episode of the first season.
• Vikings - Season 2 has begun. Political intrigue, a new wife, backstabbing and a king of Britain who doesn't run with his tail between his legs at the sight of vikings. This season certainly keeps things interesting.
• Suits - The series has never been about the law, not really, but about ritual. It was fun for a while, but every ritual, no matter how bizarre or exaggerated for dramatic purposes, can maintain outside interest only for so long. And the few law related references that made it interesting to me now are almost gone, together with the cowboyish feel of the show that made it seem less serious and more fun. Perhaps because all male characters in the series got hitched and their wings properly clipped off.
• Continuum - Weird new direction of the third season of the series, with collapsing timelines, people meeting themselves and "the freelancers" being a cult like do gooder agency that protects time. I don't know yet if I like it, but it feels more confusing and less fun and sci-fi.
• Crisis - A bunch of rich kids, including the son of the US president, are taken hostage by a shadowy mastermind who has thought of all possible outcomes before long before the actual execution and then uses the parents to achieve his goals, including more kidnapping of kids, I guess. Then there is this ex-cop Secret Service rookie black man who is set on finding the kids. So it's a kind of Die Hard, if you think about it. However the shadowy mastermind seems to have some moral agenda, revealing the evils of CIA, while the poor investigative agents are torn between their duty, their sympathy for the kids and their parents and the incredible stupid decisions people take in the name of their kids. I know Americans have this cult of parenthood and the need to do anything for your children, but too much is too much!
• Da Vinci's Demons - The Pazzi's are trying to wrestle control of Florence from the Medici's, with Rome's involvement. That leads to some deaths, rearranging of loyalties and now both Da Vinci and Reario are heading towards the Americas on different ships, using a map of the location, only 40 years before Columbus blindly went to find the continent and with the help of none other than Amerigo Vespucci, who wasn't born yet. Nor was Da Vinci's friend, Nicolo Machiavelli, for that matter. Confusing? Only if you try to align the series with history.
• Turn - This is a show about the first American spy ring, during the Revolutionary War. The first episode was rather slow, but it needed to set up the story and it showed good production values. Due to the actual nationality of the first Americans, most of the actors are British or Scottish, even if it is an AMC series. British acting with American money sounds good to me.
• The Crimson Field - A six episode British miniseries about nurses in the war. The main action happens in a field hospital where four new volunteer nurses came to help out. It seems pretty decent and I already eagerly await the second episode.
• Silicon Valley - An American sitcom about Silicon Valley, created by Mike Judge. Six programmers live in the same house when one of them invents an ingenious algorithm. Being an HBO series, with only eight episodes in the first season and created by Judge, I have high expectations from it.

Read More