Thursday, 18 February 2016

Firebase - a free Javascript/Rest accessible cloud no SQL database - Introduction

Firebase logo I have been looking for a long time for this kind of service, mainly because I wanted to monitor and persist stuff for my blog. Firebase is all of that and more and, with a free plan of 1GB, it's pretty awesome. However, as it is a no SQL database and as it can be accessed via Javascript, it may be a bit difficult to get it at first. In this post I will be talking about how to use Firebase as a traditional database using their Javascript library.

So, first off go to the main website and signup with Google. Once you do, you get a page with a 5 minute tutorial, quickstarts, examples, API docs... but you want the ultra-quick start! Copy pasted working code! So click on the Manage App button.

Take note of the URL where you are redirected. It is the one used for all data usage as well. Ok, quick test code:
var testRef = new Firebase('https://*******.firebaseio.com/test');
testRef.push({
val1: "any object you like",
val2: 1,
val3: "as long as it is not undefined or some complex type like a Date object",
val4: "think of it as JSON"
});
What this does is take that object there and save it in your database, in the "test" container. Let's say it's like a table. You can also save objects directly in the root, but I don't recommend it, as the path of the object is the only one telling you what type of object it is.

Now, in order to read inserted objects, you use events. It's a sort of reactive way of doing things that might be a little unfamiliar. For example, when you run the following piece of code, you will get after you connect all the objects you ever inserted into "test".
var testRef = new Firebase('https://*******.firebaseio.com/test');
testRef.on('child_added', function(snapshot) {
var obj = snapshot.val();
handle(obj); //do what you want with the object
});

Note that you can use either child_added or value, as the retrieve event. While 'child_added' is fired on each retrieved object, 'value' returns one snapshot containing all data items, then proceeds to fire on each added item with full snapshots. Beware!, that means if you have a million items and you do a value query, you get all of them (or at least attempt to, I think there are limits), then on the next added item you get a million and one. If you use .limitToLast(50), for example, you will get the last 50 items, then when a new one is added, you get another 50 item snapshot. In my mind, 'value' is to be used with .once(), while 'child_added' with .on(). More details in my Queries post

Just by using that, you have created a way to insert and read values from the database. Of course, you don't want to leave your database unprotected. Anyone could read or change your data this way. You need some sort of authentication. For that go to the left and click on Login & Auth, then you go to Email & Password and you configure what are the users to log in to your application. Notice that every user has a UID defined. Here is the code to use to authenticate:
var testRef = new Firebase('https://*******.firebaseio.com/test');
testRef.authWithPassword({
email : "some@email.com",
password : "password"
}, function(error, authData) {
if (error) {
console.log("Login Failed!", error);
} else {
console.log("Authenticated successfully with payload:", authData);
}
});
There is an extra step you want to take, secure your database so that it can only be accessed by logged users and for that you have to go to Security & Rules. A very simple structure to use is this:
{
"rules": {
"test": {
".read": false,
".write": false,
"$uid": {
// grants write access to the owner of this user account whose uid must exactly match the key ($uid)
".write": "auth !== null && auth.uid === $uid",
// grants read access to any user who is logged in with an email and password
".read": "auth !== null && auth.provider === 'password'"
}
}
}
}
This means that:
  1. It is forbidden to write to test directly, or to read from it
  2. It is allowed to write to test/uid (remember the user UID when you created the email/password pair) only by the user with the same uid
  3. It is allowed to read from test/uid, as long as you are authenticated in any way

Gotcha! This rule list allows you to read and write whatever you want on the root itself. Anyone could just waltz on your URL and fill your database with crap, just not in the "test" path. More than that, they can just listen to the root and get EVERYTHING that you write in. So the correct rule set is this:
{
"rules": {
".read": false,
".write": false,
"test": {
".read": false,
".write": false,
"$uid": {
// grants write access to the owner of this user account whose uid must exactly match the key ($uid)
".write": "auth !== null && auth.uid === $uid",
// grants read access to any user who is logged in with an email and password
".read": "auth !== null && auth.provider === 'password'"
}
}
}
}

In this particular case, in order to get to the path /test/$uid you can use the .child() function, like this: testRef.child(authData.uid).push(...), where authData is the object you retrieve from the authentication method and that contains your logged user's UID.

The rule system is easy to understand: use ".read"/".write" and a Javascript expression to allow or deny that operation, then add children paths and do the same. There are a lot more things you could learn about the way to authenticate: one can authenticate with Google, Twitter, Facebook, or even with custom tokens. Read more at Email & Password Authentication, User Authentication and User Based Security.

But because you want to do a dirty little hack and just make it work, here is one way:
{
"rules": {
".read": false,
".write": false,
"test": {
".read": "auth.uid == 'MyReadUser'",
".write": "auth.uid == 'MyWriteUser'"
}
}
}
This tells Firebase that no one is allowed to read/write except in /test and only if their UID is MyReadUser, MyWriteUser, respectively. In order to authenticate for this, we use this piece of code:
testRef.authWithCustomToken(token,success,error);
The handlers for success and error do the rest. In order to create the token, you need to do some cryptography, but nevermind that, there is an online JsFiddle where you can do just that without any thought. First you need a secret, for which you go into your Firebase console and click on Secrets. Click on "Show" and copy paste that secret into the JsFiddle "secret" textbox. Then enter MyReadUser/MyWriteUser in the "uid" textbox and create the token. You can then authenticate into Firebase using that ugly string that it spews out at you.

Done, now you only need to use the code. Here is an example:
var testRef = new Firebase('https://*****.firebaseio.com/test');
testRef.authWithCustomToken(token, function(err,authData) {
if (err) alert(err);
myDataRef.on('child_added', function(snapshot) {
var message = snapshot.val();
handle(message);
});
});
where token is the generated token and handle is a function that will run with each of the objects in the database.

In my case, I needed a way to write messages on the blog for users to read. I left read access on for everyone (true) and used the token idea from above to restrict writing. My html page that I run locally uses the authentication to write the messages.

There you have it. In the next post I will examine how you can query the database for specific objects.

Wednesday, 17 February 2016

Breakthrough (Breakthrough book 1), by Michael C. Grumley

book cover I wasn't expecting much from this book, as another from the same page offering free books from their authors was kind of disappointing. However, this is a true book: it is long enough, well written, with developed characters and with an end that delivers closure to all the story arcs. Indeed, it closes all of them so well that is kind of weird to see that it is part of a series, containing the same characters no less. I mean, come on, how many times can the same people save the planet? Shut up, Marvel!

It was surprising to me to find out that Breakthrough was Michael C. Grumley's debut book. It is professionally written. Nothing exceptional, mind you, but nothing you can possibly find wrong with it. And the subject of the book was complex and interesting, involving talking dolphins, undersea aliens, covert military operations (no, it is not a Seaquest ripoff), which reminded me a little of Creatures of the Abyss.

The ending was a bit rushed, I guess, and contained that annoying trope "You are not yet ready, humans!". Fuck you, aliens, if all you've got to show for your evolution are plans to either destroy or patronize us! Plus some crowd pleasing death avoidance which felt wrong. But overall it was a good book, way above what I would call average. Since it is offered for free, you can download it and read it right now. And if you like it, the author offers even more free stuff on his site.

Monday, 15 February 2016

Building robust software

the logo is from somewhere else, but it's a pic! I was reading this summary of a talk that Dr. Gerard Holzmann held at USENIX Hot Topics in System Dependability mini-conf on 7 Oct 2012 in Hollywood, California. In it there is a link to what the people in the JPL decided to use as the core of the coding standard: The Power of 10. Yeah, it sounds like a self-help system for addicts, but in fact it is a very smart idea. You see, when you code for the JPL you are talking about code that you will design and test on Earth, then run in space, often years after first developed. It needs to be robust, it needs to be as safe as possible and to make easy detecting problems early on. They tried with a style coding standard, but they failed, mostly because people were not being able to follow all the rules they decided on. Here comes the brilliant idea of taking the most risk alleviating ten coding rules and make it a kind of core of their development style. A form of software ten commandments, if you will.

Some of the rules there are quite counterintuitive. You may check them in link format here and in PDF format here. I was particularly interested in rules 2 and 3: allocate everything you need before you run the program (so eliminate things like more memory allocation or garbage collection) and giving all loops an upper bound (so make sure there will never be an infinite loop). The others are either common sense or already implemented in modern programming languages.

If I were to implement this, I would try to encapsulate the idea of finite loops, so instead of foreach/for loops I would use a class with Foreach/For methods (akin to Parallel). The memory allocation thing is trickier in .NET. The idea of garbage collector is already built into the system. The third rule in P10 says "Memory allocators, such as malloc, and garbage collectors often have unpredictable behavior that can significantly impact performance". I wonder if there is any way to quantify the performance losses coming from the framework memory allocation and garbage collection. As for disabling this behavior, I doubt it is even possible. What I could do is instantiate all classes used for data storage (all data models, basically) I will ever need at some initialization stage, then eliminating any usage of new or declaring any new objects and variables of that sort. It kind of goes against the tenets of OOP (and against P10's rule number 6, BTW), but it could be interesting to experiment with.

What do you think? Anyway, feel free to ignore my post, but read the document. People at JPL are not stupid! I loved this minimalist idea they used: just reduce all coding rules to the more important ten.

Google trend effect on page views?

Snowball Google I have implemented a system that logs what people do on my blog, with the intent of making it more useful to my readers. In doing so I created a live dashboard where people going and leaving are displayed in real time. The conclusion is pretty humbling, but I have also noticed a pattern that might reflect badly on the state of the Internet today.

The conclusion I was talking about is that, even if I write about a lot of things, from books to software, from WPF to Javascript, the most visited posts by far are about why the Bittorrent client gets stuck, how to remove ads by installing Privoxy and Sift3, my string comparison algorithm. All of that info one can get from the Popular posts column in the right of the blog, but I had no idea how many people visit it only to find how they can download their movies faster!

And then there are the programming blog posts. I am filled with pride when people open a link to learn something from my experiences. And then I see that they are looking at the posts about Crystal Reports, AjaxControlToolkit and the old ASP.Net Ajax calls. Occasionally they come for the WPF bit, which is great, but the conclusion is clear: people are mostly interested in the old posts, the ones describing older technologies that no one is talking about anywhere anymore. True, I have not posted anything significant in the last two years, but still, I feel disappointed. My blog's merit here seems to be that it is still online!

But then I realized something else. Sometimes I feel joy at seeing that a visitor opens a post that no one has opened recently. Yet, in a very short time, other people are starting to open the same link. It has happened repeatedly several days in a row, so it can't be a coincidence. And people are coming from all over: Canada, US, Brazil, Mozambique, Ghana! I can only explain it with the theory that once visited, a link increases in visibility, its Google rank goes up, thus passing a threshold that makes it appear on the first search pages. It is a snowball effect, which in part I understand and agree with, but can't stop wondering if it doesn't apply everywhere. Instead of going for the relevance that Google and other big search engines aspire towards, they cheat by treating each click as a Facebook Like! More people read it, so more people should read it, which they do, and so on and so on.

The bottom line is that I wouldn't want to see a race towards a common goal be treated as a common race towards a goal. Let all pages share the glory, rank them based on content, not the preferences of people searching for stuff. How long before Google will helpfully suggest to me to go download a movie rather than search for something for work?

Saturday, 13 February 2016

Better World (Legacy Code prequel freebie), by Autumn Kalquist

book cover I've got this from a website where several self published authors shared free e-books for promotion purposes. I chose Better World because of its description: "The last humans spent centuries searching for a new Earth. Now they face extinction.
For three hundred years, arks have carried the last remnants of humanity through dark space. The ships are old, failing, and every colonist must do their duty to ensure the fleet’s survival." Pretty cool premise, if you ask me.

Now, Better World is not accidentally a free book. It is short, ends with a "to be continued" and has no other purpose than to pull the reader into Autumn Kalquist's Legacy Code series. To me it felt a bit amateurish, which is weird. I would have thought something that would pull your audience into your work should be better edited, if not better written.

The basic plot revolves around this 18 years old girl called Maeve, a low class citizen of the colony ships fleeing Earth to find a better world. The story starts with her trying to kill herself, while a dogooder boy who is obviously hot for her stops her at the last minute. I found it interesting that the heroine of the story starts off as weak, egotistical, scared, with low self esteem and living in a world where she is pretty much powerless. Everything that - if the book were written by a guy - would have prompted people to denounce his misogynistic view of the world. Yet every writing book worth mentioning affirms that the character has to begin as powerless and defective in order to evolve. And indeed, by the end of the book you find that Maeve realizes her own strength and courage when faced with true challenges, not just with teenage angst.

However, the scenes lacked power and, whenever something interesting happened, the author introduced new characters and new ideas instead of focusing on the potential of the current situation. Maeve wants to kill herself, a savior male is introduced. She rebels against authority, a younger more naive girl is introduced in order to suffer the consequences for her. The young girl gets hurt, a love interest - another girl - appears out of nowhere to take away focus from the shame and guilt. An "enforcer", a weak minded man drunk on his policing power, is making her life hell, she reminisces about her dead parents, killed by another enforcer's decision in the past. Every single time the plot was getting close to good, something was introduced that devastated the tension and the potential and gave the reader the impression that the story evolved as Kalquist wrote, with no clear idea of who her characters were or what the final shape of the plot will be.

And then there was the climax, the moment I was waiting for, when our hero gets stuck on an unforgiving planet with her torturer as her only ally... and they just walk a little to another group of people where she shows how good she is at fixing things. So much potential down the drain. Bottom line: the author comes off as a beginner in writing, but at least she is not pretending her work is the greatest and/or puts her friends to post positive reviews. Even with this short story I could see the wheels turning smoother and smoother as I went along, which probably means her writing will improve. Unfortunately, as standalone work, Better World is not more, not less than space pulp fiction, with no real impact behind the characters or the storyline.

Thursday, 11 February 2016

The Ann Leckie Ancillary series... it sucks!

all three book covers I really missed reading a good science fiction book and when I was in this vulnerable mental state I stumbled upon this very positive review on Ars Technica recommending Ann Leckie's trilogy Ancillary. Ars Technica is one of the sites that I respect a lot for the quality of the news there, but I have to tell you that after this, my opinion of them plummeted. Let me tell you why.

The only remotely interesting characteristics of the Ancillary series is the premise - that an AI gets trapped in the body of an "ancillary" soldier that was used as only a physical extension among many others - and the original idea of using no gender when talking about people. You see, in the future, the empire language has lost its need of defining people by gender and therefore they are all she, mother, sister, etc. It is important that the genre is translated into our backward English is all female, as to balance the patriarchal bias of our society. Way to go, Ann! The books also won a ton of awards, which made me doubt myself for a full second before deciding that notorious book awards seem to be equally narrow in focus as notorious movie awards.

Unfortunately, after two books that only exposed antiquated ideas of space operas past, boring scenes and personal biases of the author, I decided to stop. I will not read the third book, the one that maybe would give me some closure as the last in the series. That should tell you how bad I think the books were. On a positive note it vaguely reminded me of Feintuch's Seafort Saga. If you want to read a similarly out of date space opera, but really good and captivating, read that one.

You see, it all happens on ships and stations, where the only thing that doesn't feel like taken from feudal stories are... wait... err... no, there is nothing remotely modern about the books. The premise gets lost on someone who focuses exclusively on the emotions of the Artificial Intelligence, rather than on their abilities or actual characteristics. If I were an AI, I would consider that discrimination. The same ideas could be put in some magical kingdom where magical people clone themselves and keep in touch. I don't know who invented this idea that the future will somehow revert to us being pompous boring nobles that care about family name, clothes, tea and saucer sets (this is from the books, I am not making it up), but enough with it! We have the Internet. And cell phones. That future will not happen! And if it would, no one cares! The main character acts like a motherly person for stupid or young people, no doubt reflecting Leckie's mood as a stay-at-home mom at the time of writing the books. You can basically kill people with impunity in this world of hers, if you are high enough on the social ladder, but swearing is frowned upon, for example.

OK, ranted enough about this. I don't care that her vision of the future sucks. I wouldn't have cared if her writing was bad - which it isn't. It's not great either, though. I do care when I get flooded with review titles like "The book series that brought space opera into the 21st century", by Annalee Newitz, or "Ancillary Justice is the mind-blowing space opera you've been needing", by Annalee Newitz, or "Why I’m Voting for Ann Leckie’s Ancillary Justice", by Justin Landon - a friend of Annalee Newitz' from the Speculative Fiction compilations, and "A mind-bending, award-winning science fiction trilogy that expertly investigates the way we live now.", by Tammy Oler, who is writing with Annalee Newitz at Bitch Media. Do you see a pattern here?

I have to admit that I think it is a feminism thing. So enamored were these girls of a story that doesn't define its characters by gender, that they loved the book. Annalee's reviews, though, are describing the wonderful universe that Leckie created, with its focus on social change and social commentary, and how it makes one think of how complex things are. I didn't get that at all. I got the typical all powerful emperor over the space empire thing, with stuck up officers believing they know everything and everything is owed to them and the "man/woman of the people" main character that shows them their wicked ways. The rest is horribly boring, not because of the lack of action, but because of the lack of consequence. I kind of think it's either a friend advertising for another or some shared feminist agenda thing.

Bottom line: regardless of my anger with out of proportion reviews, I still believe these books are not worth reading. The first one is acceptable, while the second one just fizzles. I am sure I can find something better to do with my time than to read the third. The great premise of the series is completely wasted with this author and the main character doesn't seem to be or do anything of consequence, while moving from "captain of the ship" to "social rebel" and "crime investigator" at random.

Wednesday, 10 February 2016

I presented an introduction on Reactive Extensions at ADCES

My ugly mug next to my beautiful code On the 9th of February I basically held the same talk I did at Impact Hub, only I did better, and this time presented to the ADCES group. Unbeknownst to me, my colleague there Andrei Rînea also held a similar presentation with the same organization, more than two years ago, and it is quite difficult to assume that I was not inspired by it when one notices how similar they really were :) Anyway, that means there is no way people can say they didn't get it, now! Here is his blog entry about that presentation: Bing it on, Reactive Extensions! – story, code and slides

The code, as well as a RevealJS slideshow that I didn't use the first time, can be found at Github. I also added a Javascript implementation of the same concept, using a Wikipedia service instead - since DictService doesn't support JSON.