HDRLab Blog

New Project: Bountiful Potential

Mon, 16 Dec 2019 13:59:58 +0000

Wow! It's been years since I last posted an update on this blog. If you're wondering what I've been up to, then I suggest you check out my company's website: keasigmadelta.com. Anyway, I've been working on something new. It's something completely left field for me; not tech. related at all. The idea popped into my head as I was out for a walk, and was so compelling that I had to make it happen.

It comes from the combination of three things:

The birth of my first child (who was still in Mama's womb when I got the idea)
Hearing other parents telling their kids that they're useless (e.g. "you fail at everything")
Meeting adults leading living mediocre lives because they believe the lie that they're no good at anything

Thus, the new project was born. It currently has a range of infant bodysuits/onesies with little reminders of the child's bountiful potential. The idea behind them is this: parenting comes with many challenges, and we periodically need reminders of what we're making all the sacrifices for. We want our children to live meaningful and fulfilling lives, and to turn potential into reality. That's only going to happen if we parents guide and nurture them well. The little reminders in our infant clothing can be used to check our own attitude and behaviour toward our children, so that we can continually renew our efforts to be the best example and support to them.

Bountiful Potential is still in its early stages (I've got more in mind than inspiring clothing), but it's here now. Please check it out, and let me know what you think.

Here's the link again: bountifulpotential.com.

Oh, and here's a peek:

Interested in Aerospace Engineering?

Mon, 08 Feb 2016 13:15:37 +0000

If you're interested in doing postgraduate (a.k.a., graduate for North Americans) or postdoctoral research in aerospace engineering, then you may want to check out my brother's research group at Ryerson University (Toronto, Canada):
Ryerson Aerospace Control Systems Research Group

He currently has a few openings for new researchers. This would suit someone who is interested in spacecraft control systems.

Have a look at this page, for more on the specific research areas that you could be a part of.

Heartbleed OpenSSL Bug Update

Thu, 10 Apr 2014 12:11:15 +0000

By now you should probably have heard about the “heartbleed” bug in OpenSSL. Well, this website is hosted on a server running OpenSSL. Fortunately, the web hosting provider is very quick with responding to such issues When I tested this website for the bug, I was pleased to see that it was immune.

While it is highly unlikely that this website's SSL certificate or any data was compromised, I have replaced the SSL certificate (complete with new private key) as a precautionary measure. It is recommended that you change your password now; not just on this website, but on other websites too. This can be done by visiting the “Forgot password” page, and following the instructions.

I'm Still Here; Just Incredibly Busy

Thu, 11 Apr 2013 22:06:05 +0000

There hasn't been much activity on this blog or website lately. In fact, this is the first post to this blog in almost six months! This is just a quick post to let you know that I'm still here, but just incredibly busy.

My company - Kea Sigma Delta - and its colour correction software is taking up the vast majority of my time. Starting a company is easy; it's building that company that takes a lot of effort. In the process I'm learning a lot about non-engineering stuff like marketing (which is essential for business). Sometimes it feels like I'm studying for a second PhD.

Anyway, it will likely continue to be quiet here for a while longer. To pass the time, feel free to post an interesting business success (or failure) story in the comments below.

Pyramining Referrals - Investing in Bitcoin Mining

Fri, 26 Oct 2012 03:13:49 +0000

In my previous blog post I mentioned that mining Bitcoins alone probably wasn't worth it, and suggested investing Bitcoins in a contract mining service such as Pyramining as an alternative. Pyramining is an organisation that has a very efficient Bitcoin mining infrastructure, that you can pay Bitcoins to use. Basically, you invest money and effectively borrow some of their mining hardware. Over time, that hardware earns Bitcoins via mining, and you recover your initial investment, plus a bonus.

At present (26 October 2012) the bonus is initially set at 10%, and the payback time is around 14 months (12 months to the break-even point). However, in order to invest in this scheme, you need to be referred by a sponsor. To help people out, there are referral links below. They expire over time, so I'll try to keep them updated.

Pyramining Referral Links

What is Bitcoin and Bitcoin Mining?

Please see my previous post.

IMPORTANT: Any money that you invest is entirely at your own risk.

1683nt8XapRxRtXHkCqHqrkJfV5z1Av5Bu

Bitcoin - A Digital Currency

Fri, 26 Oct 2012 02:09:09 +0000

Bitcoin is a digital currency that is designed to work like cash. I first heard about it in the June 2012 edition of the IEEE Spectrum magazine. The idea of a digital currency that was not centrally controlled and worked like cash intrigued me, and the fact that it appeared in the primary magazine of the Institute of Electrical and Electronics Engineers added credibility. So, I decided to check it out.

How BitcoIn Works in Brief

Bitcoin is built primarily on two technologies: peer-to-peer networking, and cryptography. Peer-to-peer networking ensures that the system is fully decentralised, so that no one central authority can control it. Cryptography makes sure that transactions can be both ananymous (like cash), and secure. I'll explain the basics here. Those who want the technical details can go here.

Mining for Bitcoins

A Bitcoin miner is a computer/machine that calculate cryptographic hashes to validate blocks. Miners are rewarded for creating valid blocks with a set amount of Bitcoins (currently 50), and any transaction fees in that block. The difficulty level of validating blocks is adjusted such that valid blocks are created at a set rate. This is done so that no-one can print money, and it is also the basis by which the Bitcoin network is kept secure.

There is a catch with the mining; only the miner that finds the solution earns the 50 Bitcoins for that block. The more processing power that a miner has, the more likely it is that he/she will get the reward.

This process might sound strange, but it will become clear later.

Sending and Receiving Money

Each Bitcoin user has a wallet containing their Bitcoins. These coins are transferred between one user and another via public and private keys (of which each wallet can have many). To send money to another user, a transaction is created to one of their public keys (a.k.a., an address), and is signed by the sender using a private key. Only the sender knows the private key, but everyone else can use the public keys to verify the transaction.

At this point, the Bitcoin miners come into play. Transactions are assembled into blocks every ten minutes, and the miners validate each block, and add it to an ever increasing block-chain. This locks the transactions into place. If some nasty person wanted to change a transaction (e.g., to redirect the money to their pocket, or spend money twice), they would have to revalidate not only the block containing the transaction, but every block that has been validated since then. Thanks to the sheer size of the total processing power of all miners in the Bitcoin network, the processing power required to achieve this is so immense, as to be almost completely impossible. Hence, the transactions are irreversible, anonymous, and yet the system is secure (unless someone can control huge amounts of processing power).

How Can I Become a Bitcoin Miner?

Woah there, slow down a bit. Yes, it sounds like a dream come true to be able to earn money simply by running a computer program. However, remember how I said that the difficulty level for validating blocks (and hence, earning Bitcoins) is adjusted to keep block production constant? Well, there are lots of others with exactly the same idea, so the cost of mining tends (i.e., the cost of the hardware and electricity) to be at or just above the amount earned.

Bitcoin mining is very competitive. Initially, a powerful CPU was enough, then as more miners joined in, powerful GPUs were required. Now, it's more cost-effective to use FPGAs, and soon, ASICs will provide an even more energy efficient way to mine. It is unlikely that you will earn much of anything by yourself.

If you do want to try Bitcoin mining, then you may want to consider using a mining contractor like Pyramining instead of investing in your own mining rig.

How do I Use Bitcoin?

First you need to download a Bitcoin client/wallet. I suggest using something like Electrum, as it's faster than the original client.

EDIT (Feb. 2021): I'd recommend buying a hardware walled instead downloading a Bitcoin client.

Next, you need to put some coins into your new wallet. There are a few places where you can get some free (micro) Bitcoins; just search for "free Bitcoins" on the internet. Or, you can do things the normal way, and use a Bitcoin exchange to transfer money from standard currencies - such as dollars - to Bitcoins. These currency exchanges are bidirectional, so you can also exchange Bitcoins back to your local currency.

Is Bitcoin Worth Using?

I'm not 100% convinced that it is worth using yet. There are some advantages:

The currency is decentralised, so no one central authority can take control
Since there is no central authority, transaction fees are lower
Transactions are as anonymous as cash
Transactions are irreversible (i.e., no credit-card charge reversals, which is good if you're the seller)

but, there are also disadvantages:

Transactions take approx. 10 minutes to be validated
If someone can hack into your computer they might be able to steal your coins remotely (NOTE: This risk can be minimised by encrypting your wallet)
Transactions are irreversible (bad, if you're a buyer who has been scammed)
The anonymity can be abused by those with low moral and ethical standards (just like physical cash)

The more people who use the currency, the more useful it becomes. Right now, the number of Bitcoin users is steadily increasing, and it looks like it could reach critical mass. Real transactions are taking place, and real items are being bought using this currency. However, there are no guarantees. Maybe it will become widely accepted; maybe something else will take its place. Or, maybe governments and financial institutions will try to shut it down because they can't control it. Who knows? Nothing is without risk.

For now, I'll say that it is an interesting idea that looks like it has a good chance of succeeding. It's definitely something worth keeping an eye on.

P.S., If you want to test out sending money using Bitcoin, you can send me a donation via this address: bc1q3zw3dax70lcdmfhjynh8qwd70wmsnfqf4vesz6

It'll be gratefully received. ;-)

Relight - Photo Colour Correction Software

Thu, 18 Oct 2012 16:30:13 +0000

Back in February, I announced that I had founded a company, and was looking for beta testers for its first product. I've been hard at work since then, I am pleased to say that this first product is now available. Called Relight, the software is designed to fix the lighting/colour issues in photos quickly and easily. It specialises in this one task, making the user interface easy to understand and learn (unlike heavyweight image processing applications with their myriads of controls). For example, the photo on the right was taken indoors, and ended up looking rather red. Using Relight, this was corrected to produce the enhanced photo below it, and it only took a few seconds to do. As you can see, the colours have been corrected, and now look the way that they should.

While I'm pleased to have reached this stage, I already have plenty of ideas for enhancing Relight, for version 2 and beyond. There is plenty more that I would like it to do (while still remaining easy to use, of course). Having said that, I would be very interested to hear what other people think, and what they would like to see in a program such as this. So, if you have tried out Relight and have any suggestions or comments, please send a message via the company's contact page.

P.S., If you're quick then you can get a 41% discount. You'll have to hurry though, because the introductory offer ends 4 November 2012 (New Zealand time).

A New Venture

Thu, 16 Feb 2012 16:06:32 +0000

I have a small confession to make: I had an ulterior motive when creating this website. Back then, I knew that I wanted to start and run my own company one day, and that the company's website would be a very important part of the overall system. In order to prepare for this, I wanted to learn how to run a website with online services; both the technical and human aspects (websites are, after all designed to be used by humans). So, this website served as the platform by which I could learn and experiment with building and running a website.

I am pleased to announce that my company, Kea Sigma Delta Limited, was incorporated in November 2011, and is now seeking beta testers for its first product. The first product will be a photo enhancement tool that is designed to help easily correct colour/lighting issues in photographs. If you have an interest in photography, and would like to help test this application, please follow the instructions to register your interest.

A lot of work has gone into getting this far, and there is still much work to be done before the company is fully up and running. Incorporating a company is easy; building a company is not. Nevertheless, it is exciting to get to this stage, and I am looking forward to eventually releasing the first product.

While this new venture will be taking up most of my time (as it already has been for a while), HDRLab will still be operational. This is my personal website, and I intend to keep updating it. I will also be continuing to work on the various projects detailed here (time permitting).

Login to HDRLab Using your Facebook Account

Thu, 18 Aug 2011 00:21:41 +0000

It is now possible to log in to this website and post on the forums using your Facebook account. This was added so that people don't have to create and remember yet another user and password combination. Like many people, I have an increasing number of accounts on multiple websites, and remembering all of those usernames and passwords gets harder and harder (particularly with websites that have "special" rules regarding what a password should have). Special thanks to Will Rossiter for developing the Silverstripe Facebook Connect module.

To log in using Facebook, simply click on the "Login with Facebook" button at the top of the forums, and enter your facebook login in the window that opens. If you already have an account on this website using the same email address as your facebook account, then your existing account will automatically be linked to your Facebook account. Otherwise, a new account will be created for you.

NOTE: New accounts created when logging in using Facebook will use your first name as forum "nickname" by default. You can edit your profile at any time using the "Profile" link shown next to the "Log Out" button at the top of the forums pages.

GfxBench2D's Traffic Spike

Thu, 21 Jul 2011 02:41:09 +0000

I haven't posted to this blog in a long while, so I thought that I would share some interesting traffic data related to the newest section of the website. Just over a month ago I released GfxBench2D, a 2D benchmarking tool and web-application. This tool measures 2D graphics performance and can (optionally) upload results to this website for display. Well, this triggered the biggest traffic spike that this website has ever seen. Take a look a graph of the number of daily visits that have occurred since this website was first created (up to 10 July 2011):

As you can see, it dwarfs the next highest traffic spike, which coincided with the announcement that compositing support for Radeon HD 2000-4000 cards had been added to the RadeonHD driver. Not only did it dwarf all previous traffic spikes, but it was also the first spike not generated by the RadeonHD driver project (the first spike was the very first entry to the RadeonHD Development Log). The crazy thing is that this spike in traffic could have been even higher if I had submitted the news simultaneously to the various AmigaOS related forums. I deliberately staggered the news posts to spread the traffic out over several days. The graph of unique visitors follows the same pattern. Clearly, GfxBench2D appeals to a lot more people than than the Radeon HD driver project does despite only an AmigaOS 4.x version of the tool being available.

The effect of GfxBench2D's release becomes even more pronounced when looking at the number of page-views per day:

Not only did more visitors check out GfxBench2D than any of the RadeonHD development log posts, but they also collectively viewed more pages while they were here. This difference comes as no surprise. While all of the RadeonHD development log posts are well contained within a single page, the GfxBench2D section provides pages and pages of results (182 results at present) to examine. As a result, visitors tended to browse through the available benchmark data instead of reading a single page and then leaving.

The final thing that is noticeable in the graphs above is the slow but relatively consistent rise in traffic to this website. This is a trend that I hope will continue or, preferably, will be able to accelerate.

Website Search Feature Restored

Sat, 23 Oct 2010 00:00:00 +0000

I got an email this morning from someone who tried to use the search functionality of this website, and got a "Generated with the default ContentController.ss template" error. Apparently, the search function has been broken since I upgraded to Silverstripe 2.4.0, and no-one noticed (obviously, I didn't), or bothered to tell me up to now. Anyway, it is now fixed, and I have updated the Silverstripe JRank Integration page with a new archive.

There were two issues. The first is that the API of Silverstripe's XML class was changed, breaking my code in the process. However, even with this fixed, the Generated with the default ContentController.ss template" error persisted. This was despite the template being present, and even being partially executed (according to stack dumps that I deliberately triggered). This confused me for a while. Even more confusing was that it suddenly worked, albeit with a broken layout, when I changed the SearchResultsPage class to be a child-class of Page, and its controller to a child-class of Page_Controller.

Eventually, I realised that the template file was in the wrong place. Since the SearchResultsPage is a direct child-class of SiteTree, its template is supposed to be in the templates directory (where the base Page.ss is), not templates/Layout, where it had been put. For some reason having the template in the wrong place used to work pre-Silverstripe-2.4.0, but not anymore.

NOTE: I made the SearchResultsPage class a child-class of SiteTree so that I could eliminate the left-column navigation bar for the search results, which would have made the layout too cramped. If you want to use the same base layout as the rest of your pages, change the SearchResultsPage to be a child-class of Page, the SearchResultsPage_Controller to be a child-class of Page_Controller, and move SearchResultsPage.ss to templates/Layout.

Heater Recommentation for a Cleaner, Healthier Room

Fri, 03 Sep 2010 18:26:10 +0000

This post may seem a bit out of place in this blog, but seeing as it relates to health, I think that it's important enough to post anyway. It is also an example of how good design and attention to small details makes a difference. The short story is that I recently bought a new heater, and it has done wonders to the air quality in my room. One small change has made the room so much more pleasant to work in.

The Problem - Poor Air Quality when Heating the Room

I do most of my work in a small room without central heating, and had been using a small fan heater to stay warm during winter. While the warmth was welcome, the air in the room quickly became very stuffy. It seemed like the heater was sucking up dust off the floor, heating it, and then blowing it into the room, sweeping even more dust into the air. Yuck! As it turns out, this was more or less exactly what was happening.

Needless to say, it wasn't particularly pleasant breathing in that heavy dust/particle laden air, and I frequently opened the window just to flush it out again. Opening the heater to clean it out, and vacuuming the room made a little difference, but it still wasn't good enough. Since I spend a lot of time in that room, it was time to find a new heater. One that would heat the room without the stuffiness.

The Solution - DéLonghi Ceramic Heater

The DéLonghi TCH7592PB
Ceramic Heater

Having decided that I needed a new heater, I set about looking for one that would keep the air clean and breathable. The room was too small to bother with a heat-pump, so the search was on for a conventional heater. Whatever it was, it had to have the following two features:

An air/dust filter,
Relatively quiet, and
An electronic thermostat.

To my surprise, almost no heater had a dust filter on its intake, something that would help reduce dust particles in the air. After much searching, I found that the best option was the DéLonghi Ceramic Heater (NOTE: Model numbers may vary from country to country, in New Zealand, it's the DéLonghi TCH7592PB, whereas in the USA, the closest model is the DéLonghi TCH7590EB). I had never thought that I would spend $170 on a heater for a small room, but it has been worth every penny. The difference to the room's air quality was so big, that I wondered how I had put up with the old heater for so long. Keep reading to see why.

Good Design - Attention to Detail

So why do I think that this heater makes such a big difference. It is a combination of attention to detail and the quality. Firstly, the heater stands tall vertically, moving both the air intake and outtake off the floor. As a result, it isn't sucking dust off the floor, or whipping dust of the floor/carpet into the air. This one design decision alone reduces the amount of dust in the air. Next, it has an air filter which, despite being a small and cheap addition, nevertheless helps keep the air in the room clean. Having the air filter is one of the things that separates it from other tall heaters. Those of you who are concerned that a tall heater might fall over and burn the house down, don't be; there is a switch in the base that disconnects power to the heater if it tilts over (or is lifted off the floor). There are other vertical orientated heaters out there, but this is the only one that I found that had an air/dust filter.

It is also quiet. No, not Mitsubishi heat-pump quiet, but much quieter than the rumbling that most fan heaters make. In low heat mode the fan is slowed down, and only a soft air-through-pipe noise is heard. In automatic mode, it will heat the room rapidly, and then switch to the quieter low mode one the room is close to the desired temperature.

The other important point is that it is good qulity both in design, and construction. Despite having a lot of plastic parts, it is solidly built, and doesn't look like it's going to break easily. The plastic is also of a type that can handle the heat. Due to its vertical design it also doesn't take up much space.

The heater has other features, but they aren't important to me, and didn't factor in my buying decision. By far the most important thing is that I now have a warm room without the stuffiness. Breathing clean air makes a world of difference.

I recommend anyone who is using cheap fan heaters to consider replacing it with the DéLonghi Ceramic Heater, or any other heater that has an air/dust filter. If you don't mind waiting longer for the room to heat up, a convection heater that's easy to clean might be another idea. Just ditch that cheap fan heater, your lungs will thank you for it.

Computer Dies, but it's Only the Power Supply

Thu, 10 Jun 2010 02:03:36 +0000

I got a bit of a scare today when my AmigaOne XE G4 computer stopped working. Unlike standard PCs, this computer cannot be replaced by simply walking down to a computer store and buying another machine. If this machine did have to be replaced, I would have to order one from overseas. Since I am hard at work with various Amiga projects (particularly this one), this would have been quite a setback.

I have had trouble switching it on for a while now, but thought that this was due to a flaky power switch. As it turned out, it wasn't. The power supply was nearing its end. Today, I was alerted to a problem a beeping noise that sounded like a wrist-watch alarm. Looking at my Amiga, I saw that it was coming from the fan controller, which showed that the temperature was 60 degrees, and rising. So, I quickly shut it down. After opening the case and waiting a minute, I tried to switch it back on, but it failed. A few more attempts later, the fans switched on, but rotated very slowly and made a horrible noise. The power LED remained off. Clearly, something was broken.

Since I was already suspecting the power switch, I pulled the power switch out, and performed some quick tests with a multi-meter. Sure enough, the switch was perfectly fine. Besides, the switch could not have caused the strange fan behaviour that was observed.

By this stage I was thinking in dismay about how long it would take for a replacement machine to arrive. Added to that, I'd rather not spend money on a new computer until I can get my hands on an AmigaOne X1000. Anyway, given the way that the fans were sort of turning, but making a dirty rasping noise, I started suspecting that the power supply might have died. Fortunately I happened to have a spare 230W ATX power supply lying around (was going to be a power supply for an electronic project; not any more), so I swapped the power supply around, plugged the machine back in, and pressed the power button. The machine immediately started; no trouble with the power button, no noisy fans, just a computer working as it should. Phew, that was close.

This incident has reminded me just how disruptive hardware failure can be. In this case, shipping a replacement Amiga would take weeks. However, the last time that I had to reinstall Windows on my laptop (after the audio stopped working), it took me two days before I had everything reinstalled, and back to approximately how I wanted it. First, a restore from backups on a network drive failed, then the Windows install CD failed due to some mysterious problem with the DVD drive. Finally, a copy of Windows XP x64 installed, but it still took a very long time to reinstall all the software. I cannot remember if it was Norton Ghost or Acronis True Image that failed with restoring from backups. All I know is that neither program has worked properly for me, and so I'm still facing the possibility of a lengthy reinstall the next time that disaster strikes. This is a problem despite my rigorous backups of all critical data.

Maybe a better idea would be to have a system that backs up only the user data, and keeps a record of what programs were installed with what settings. Then, instead of attempting a whole system restore (which seems to fail a lot), the restore utility would run the appropriate installers to get a fresh copy of all applications, and then restore the user's data. In theory, this should be more reliable, result in a more stable reinstalled system, and save a lot of space with the backups. To date I have never found a product that works like this, so this is a free idea for anyone working in the data backup industry.

In the meantime, all my machines are once again working as they should.

Upgraded to Silverstripe 2.4.0

Sun, 30 May 2010 15:49:08 +0000

This website has just been upgraded to Silverstripe 2.4.0, which adds (amongst other things) hierarchical URLs. While the effort has been worth it, this upgrade was more problematic than previous ones due to version 2.4 including a cleanup of the API, and some rather major changes. Fortunately I now have a test server that I can do trial runs on before making changes to the live website. Here's a quick rundown of the problems that I experienced, and the solutions.

With the first test upgrade, a series of warnings about extraStatics() being deprectated, followed by an error saying: [User Warning] Cookie 'bypassStaticCache' can't be set. After a quick look for this error on the Silverstripe forums, I discovered that a set of modules had to be upgraded. Thus, the following modules were upgraded to either the latest public release (if that release supported Silverstripe 2.4), or the latest daily snapshot:

auth_openid,
blog,
forum,
gallery,
mollom,
newsletter,
securefiles,
spamprotection, and
userforms.

With these modules upgraded, the rebuild (/dev/build) succeeded. However, new problems arose due to some module upgrades being incompatible with older versions. In particular, the forum module had lost all posts, and some of its links (e.g., the login link) were broken. The broken links were due to the templates being out of date, so I took the new templates, and fixed up my theme's CSS files so that the forum looked good again. Next, the forum documentation mentioned a migration task (/dev/tasks/ForumMigrationTask). After executing the migration task, all of the forum posts reappeared.

The next issue was with the userforms; the form itself had disappeared, leaving only the submit button. Here too, a migration script was available. Unfortunately it didn't work. It complained about cartain database tables starting with _obsolete were missing. Looking in the database (by executing 'SHOW TABLES;' in MySQL), I noticed that the tables in question had been renamed with all lowercase, and the script was expecting certain letters to be capitalized. So, the script was updated to match the database, and the script executed a bit further. Nevertheless, there was still an error: "Unknown column 'CustomParameter' in 'field list'." Another search on the forums revealed that this was due to a residual obsolete column still being present. Executing the following fixed it:

ALTER TABLE `EditableFormField` DROP `CustomParameter`;

With this done, the migration script completed successfully, and after republishing the forms, the forms worked correctly again.

The last remaining issue was very minor; the blog widgets had disappeared. These were simply reinserted using the Silverstripe CMS pages. Unfortunately the Tag Cloud and Archive widgets are still not working. These are issues that I still need to look in to.

Once the website was working properly on my test server, the upgrade procedure was repeated on this live site. Interestingly, data (e.g., form submissions) that had disappeared on the test site remained intact in the actual upgrade. This is probably due to various steps that I took back then being skipped in the actual upgrade, because I knew by then that they would not work.

So far Silverstripe 2.4.0 is performing well. Small issues that I had previously are now gone, it feels a little faster (although this may be pure imagination), and the hierarchical URLs are a welcome improvement. Don't worry, the old URLs are automatically redirected to the new ones so existing bookmarks should still work.

Adbrite Dumped for Poor Performance

Tue, 27 Apr 2010 00:00:00 +0000

It's been a while since I have posted anything to this blog, mainly because I have been busy with other things. I have, however, still kept an eye on it, deleting spam when necessary, and monitoring how the advertising programs are performing. Today I made the decision to dump Adbrite.

A few months ago, Abrite revenue dropped to almost nothing (over a 10 times drop), and stayed there. I decided to give it a few months before deciding whether to continue using their service or not. Well, revenue has remained very low, and a pitiful few impressions are actually shown. Adbrite advertisements also had a poor click-through rate due to poor relevance. Hence, today I ditched that advertising network.

Ironically, round the time that the sudden unexplained revenue drop occured, I was about to ditch ADSDAQ for providing mostly ugly slimming and teeth whitening advertisements, and stick with Adbrite. Since then, has ADSDAQ improved dramatically, and filtering has removed the offending advertisements.

Acording to this article, someone else had a similar dramatic drop in revenue from Adbrite after they limited his accound for invalid clicks without any notification. This may have happened in my case, because the month prior to the revenue collapse my account recorded a few clicks (yes, performance was that poor). I am not going to bother checking if this is the case because I would still dump Adbrite for the following reasons:

Failing to notify customers/partners of important events (such as restricting their account for alleged violations of terms and services) is completely unacceptable business practise to me, and
If my account has been restricted for alleged "invalid clicks" which I did not make or cause, that would demonstrate a very poor invalid click detection algorithm on which I cannot rely, whether I am advertising, or providing advertising space.

So, either Adbrite has suddenly started performing very poorly, or their "invalid clicks" technology and business practises are unacceptable. Either way, they're gone.

I still haven't found an advertising network that I am satisfied with. ADSDAQ is okay, but has a low fill-rate (impressions per page view). Chitika works quite well, but caters only to visitors from the USA and Canada. Google Adsense was okay, until they disabled my account due to invalid click activity from an unknown source (I still have no idea who was responsible). I have just started testing Brinkin Banner Exchange, which combines concepts from both banner exchanges, and contextual advertising. It is an interesting concept; we shall see how it performs.

NOTE: Brinkin suddenly just disappeared a long while ago, and so I can only assume that it didn't make enough money in order to survive. (2012/03/9)

Mollom Learns from its Mistakes

Mon, 07 Dec 2009 00:00:00 +0000

Several weeks ago I mentioned that spammers had found a weakness in Mollom. Well, Mollom's learning algorithm has successfully learnt the new tricks that spammers were using, and the spam comments have stopped. The number of spam attempts has not decreased, but I have now had over a week without a single spam comment coming through.

Mollom's anti-spam algorithm learnt the new behaviour fairly quickly. Within a few days, the number of spam comments that made it through dropped from over 100 down to only a few. Getting rid of the last few remaining spam comments took another few weeks. I still hope that Mollom will learn faster in future, but it's current performance is quite good.

Fighting with Linux

Fri, 27 Nov 2009 00:00:00 +0000

Last night I had bit of a battle with my Linux server. The evening had already started out badly with a backup of this website failing due to some mysterious Sqlite error (note, that is was on a Windows machine). Next, backups from my laptop to the server failed, and I could suddenly no longer access network shares on the server by the server's name. So, I logged on to the server from my laptop, opened the Samba config file, and added an entry, specifying its netbios name, and then restarted Samba. At this point, the server was visible in the "network neighbourhood" again, and the backups could proceed. Little did I know that this was only the first problem that I was going to be dealing with that evening.

With the backups working, I turned my attention to the little indicator that said that there were system updates available. "Might as well update those files while I'm here," I thought. Bad idea; had I not done this, my evening would have been much more enjoyable. One by one all of the updates were failing, due to security.debian.org not being accessible. I had noticed this before, and decided to fix this issue. Security.debian.org was actually accessible; it responded to pings, and the web browser had no problems accessing it. After searching the internet, I discovered that you disabling ipv6 solved the problem (I've written a how-to, here, for future reference). With that done, the updates started coming. At the same time I decided to upgrade to the latest version of Debian.

Upgrading a Debian distribution takes a long time, so I left my machine running, and went off to more fun stuff. Returning late in the evening, the remote Gnome session window on my laptop, had gone black, and wouldn't wake up. Normally, I would just close the session and start a new one but, with an update in progress, I did not want to do this. After another search on the internet for ideas, I eventually discovered that one can regain access by killing the screen saver.

My joy at having learnt a new trick was short-lived. The system update had stalled a while ago because one of the updates had a question for me. Grumbling, I pushed enter, and then waited; I knew that there would be many more questions during the update process. Eventually, updates started failing. Security.debian.org was un accessible, again. A system reboot failed to make a difference. This was rather frustrating since it was late, and I thought that I had fixed this problem. Pinging the server gave the answer; this time server.debian.org really was down. Argh!

Security.debian.org remained down till mid-morning today. Now the system is completely up to date, and I'm relying on Debian's reputation for reliability to save me from having another frustrating battle like this for a while. Ah, computers; so much more complicated then they need to be.

WidgetBucks Removed from This Site

Wed, 25 Nov 2009 00:00:00 +0000

I've been trying out different advertisers on this website, WidgetBucks being one of them. The hope is that, one day, this website will cover its own hosting costs. As of yesterday, WidgetBucks is no longer active on this website. This was done because WidgetBucks popped up a message saying that they were going to disable the account in three days because it was failing to meet "minimum requirements," as outlined in the terms and conditions. I checked the terms and conditions, and there was nothing there that I had failed to meet; they didn't even have minimum revenue requirements. Rather than try to fight the decision, I decided to pull all of their advertisements off this site for the following reasons:

The revenue was extremely poor, averaging just one click per month,
WidgetBucks is very North America centric, whereas this website's visitors are spread out globally,
Visitors from most countries get very low quality CPM (Cost per thousand impression) advertisements (e.g., those annoying flashing "you are the 1,000,000th person to see this" advertisements), and
Most advertisements were completely irrelevant to my website's content, and thus served only as a waste of space.

Given all of the above, I decided that WidgetBucks was not worth continuing with. It may work well for sites with primarily North American visitors, who are interested in buying MP3 players and/or printer cartriges, but for other sites, it is likely to perform poorly. Irrelevance was the same reason that I ditched Bidvertiser months ago.

At present, this website still uses two advertising services, Adbrite, and Chitika. Adbrite's advertisements are still marginally relevant but, I can (and do) reject advertisements, and people can purchase advertisements to run on this website simply by clicking the "Your Ad Here" link below the banners.

Most visitors will probably have never seen Chitika advertisements, because they only show up when someone from a supported country (currently only the USA and Canada, but they plan to expand) arrives at a page via a serach engine. The advertisements are tailored to what they are seraching for (so it's likely to be relevant to them). Everyone else (including regular visitors) don't see the advertisement. Despite showing only to American and Canadian visitors who arrived via search engines, Chitika is currently outperforming Adbrite. This is most likely due to the advertisements being shown by Chitika being relevant to what visitors are searching for.

This website is still far from covering its own server costs from advertising. With an average of about 100 page views per day, this is not surprising. Daily page views should be in the thousands or higher in order for advertising revenues to reach a meaningful level.

AVLabs Digital Photo Frame Stops at "Initializing"

Tue, 24 Nov 2009 00:00:00 +0000

If you have a digital photo frame from AVLabs (or some other company) and it simply shows "initializing" when you insert an SD card, and switch it on, try a smaller card. I came across this problem recently when my sister bought a digital photo frame, and a 4 GB SD card. Unbeknownst to both of us, a 4 GB SD card, is in fact a 4 GB SDHC (High Capacity) card, and the SDHC standard is incompatible with the SD card that it is based upon.Yes, that's right, despite having SD written on the front (with HC in a different font), and having the same physical size as the original SD card standard, only SDHC card readers can access the data on 4 GB to 32 GB SDHC cards.

After taking the photo frame and SD card home, photos were copied onto the SD card, the card was inserted into the photo frame's card slot, and the device was switched on. The result was a black screen with "Initializing" written in the top left hand corner. The photoframe could still be switched to AV input, but otherwise it remained "Initializing" perpetually. Unfortunately, none of those who had this problem seem to have fixed it (and posted it on the internet for the benefit of others). This blog post aims to rectify this.

At this stage, we had a quick look through the manual, which was about 16 useless pages of stating the obvious, and providing no useful details (stating that the switch marked power switches the device on and off is not useful). Next, a quick look on the internet, brought up conflicting reviews; some said that it was a great device, while others complained about the poor manual (agreed), and how it simply says "initializing." Okay, others have the same problem.

Next, I tried a USB drive, which it also failed to read. After a few wasted hours, it was discovered that it would read a USB stick, but not a USB hard-drive. Finally, I looked at the Secure Digital entry in Wikipedia, which states:

"Changes to the interface of the established format have made some older devices designed for standard SD cards (≤4GB) unable to handle newer formats such as SDHC (≥4GB). All SD-cards have the same physical shape and form factor however, which causes confusion for many consumers."

Ah, so that is the problem. After copying the photos to a 2 GB SD card (minus the HC bit), everything worked perfectly.

To me, this is an example of poor user interface design. Sticking at "Initializing" when a device isn't readable is very poor feedback to the user; it tells the user nothing about why it doesn't work, and makes it very likely that he/she will take the device back to the shop. Having a user manual that doesn't provide even basic setup/installation instructions only adds to the problem. Granted, when a photo frame works, there isn't much to the installation but, when things go wrong, the manual should provide some assistance.

In summary:

Some digital photo frames (e.g., the one that we tried from AVLabs) cannot handle SDHC cards or higher (i.e., cards with 4 GB or more storage space),
Some photo frames can also only handle USB sticks, not other kinds of USB storage devices, and
If it doesn't work with your existing card (e.g., stops at "Initializing"), try a 2 GB SD card or smaller.

Software-as-a-Service (SaaS) Challenges - Reliability and Trust

Wed, 18 Nov 2009 00:00:00 +0000

I have always had my reservations about Software-as-a-Service (SaaS) due to issues such as reliability and control over my own data. To date, no SaaS vendor has addressed my concerns, indeed some people have been openly agressive, to the point of suggesting that I'm so paranoid that I need my head checked. Well, last night I had a problem that highlighted my reliability concerns: the power supply to my home router broke, leaving me without internet until I could buy a replacement today. Due to Dick Smith electronics becoming increasingly poor in its electronics selection (it is now hopeless for the electronics enthusiast), I had to drive all the way into the other side of downtown wellington to Jaycar, in order to buy a 5VDC 3A power supply. One small failure knocked out my internet connectivity for half a day.

I'll get back to my power supply failure, and SaaS reliability soon, but, for those who aren't familiar with the term, SaaS desrcibes software that is accessed as a service over the internet, rather than buying and installing it on ones own machine(s). From the SaaS vendor's perspective, it's great because:

You have a continuous stream of revenue from subscribers rather than a one time revenue when people buy, and
Pirating the software is almost impossible (bar security breaches) because customers never have the full application; the application's core functionality runs on the server, and the customer only gets the display/user-interface portion.

Reliability

Back to my power supply failure. The power supply is but one small part of the chain between me, and the SaaS system that have to work properly. Depending on where the servers are lkocated relative to me, there are

my computer,
the router (and its power supply),
the DSL modem (if it's separate),
the DSL hardware at the local exchange,
the network connection from the local exchange to the ISP's nearest hub,
the connection from the ISP's hub to the national backbone,
the connections from the national backbone to the server's country (may go through multiple countries), and all routing equipment involved (possibly including satellites),
the server's machines, and
implicitly the power generators, power lines, sub-stations and power supplies that supplies all of the above with power.

If any of these systems fail, then so to does my connection to the server. Sure, each of these systems is very reliable, but the more serially connected systems that you have, the greater the likelyhood of overall system failure. Putting multiple systems in parallel, increases redundancy, and thus, reliability (which is why we backup our data).

Now, let us assume that I was using a service such as Xero, and/or PlanHQ as a critical part of my (fictitious) business. If any critical device fails between their severs, and my business fails, then I can no longer access and update my accounting data (Xero) and/or my business planning and collaboration data (PlanHQ). Both services allow you to save the data in some external format, but is this enough? It still won't be possible to update or modify the information. One could say, "just wait until it's back up," but that does not address the issue. If the failure lay outside of the business, then I would be able to do nothing to make the service come back faster, other than maybe yelling at the vendor, and hoping that this helps. Moreover, there would be that annoying realization that I would have had none of these issues if the server was running within your own organization (and if it did fail, I could fix it).

Things would be even worse if my company also used salesforce.com; then the entire sales team would be unable to do their work effectively. If the eCommerce website also used PayPal, or some other payment gateway then sales would also grind to a halt. The more external systems that the business relies upon, the more likely it will be that something fails, and takes part of the business down. Don't think that it can happen? My power supply failure is a small example of the many things that can go wrong. Other examples include:

Earlier in this decade a major systems failure (I cannot find the details) at the moment) of one of the major data hubs on the USA's west-coast crippled New Zealand's main data link to the rest of the world for several days,
In 2003 the entire north-east of Canada and the USA had a power blackout caused by one fault triggering a cascade of shutdowns (not directly data related, but servers need power); other blackouts have occured around the world and in New Zealand too, over the years,
In 2008, a Pakistani telecommunications company inadvertently knocked youtube offline globally,
In June 2009, Xero was taken offline by a massive data center failure in the USA, and
In August 2009, Paypal went offline for several hours (for the second time this year),

Failures can, and do happen, despite the internet increasingly haing multiple connections and redundant systems.I do have to point out that, in the case of a failure, SaaS vendors may be in a better position to deal with it then a small company would, if that small company were maintaining its own systems. Nevertheless, there are still many more things that could go wrong, many of which are outside the control of both the customer, and the vendor (i.e., anything that happens on systems between the two).

Data Security and Trust

The next major issue is whether or not one wishes to entrust private data to a third party's care. When I mention this, invariably someone pipes in with "if you're doing nothing illegal, then what is the issue?" Cue, accusations of paranoia. This is very easy to say if we're talking about a few personal photos and you live in a country with freedom of speech, and in which persecution of political or religious views is illegal, but it misses the wider picture; maintaining access to ones own data is just as critical as preventing others from reading and using it. Here the reliability issue raises its head again; all that personal/business data is of no use to me if I cannot access it myself. Moreover, Microsoft's computing cloud demonstrated vividly that a computing cloud can lose data when its Sidekick service lost T-Mobile's customers emails, and personal data. A computing cloud is supposed to be highly redundant, and robust to failure, yet here was a massive data failure that no doubt resulted in the loss of highly critical information. This is not the only data failure that has occurred.

Next is the issue of protecting sensitive information. For many businesses their information is a critical part of their competitive edge, so maintaining privacy is critical. Customer's personal details are particularly sensitive, and having that data stolen has serious consequences. Fortunately, this also applies to SaaS companies, so it is in their best interests to make sure that no-one - not even they - can access your data. SaaS companies typically have stringent privacy policies, but this does not necessarily translate to technological barriers such as encrypting customer data. For example, Xero's privacy policy is good, but it does not explicitly say how securely the data is stored; it states that Xero employees cannot access usernames and passwords, and so cannot access customer data without the customer's administrator explicitly giving them permission, but would someone with physical access to the server be able to read the data. Why would someone want to do this? Well, if your company developed some technology that a competitor felt would bankrupt them, then there would be ample motive to have a go at the data center that it was stored on. Most of us will never be in a position where this matters, but it is still something that SaaS companies have to address. One only has to look at the myriad of credit card number theft reports to realize that privacy policies does not equal data security.

I have to point out that data on an in-house server is just as susceptible to theft as data on an SaaS company's servers; in fact, it may be even more so if your company lacks data security experties. However, this is another issue that makes people like me more reluctant to use SaaS software. Plus, keeping the applications and data in-house just feels better, because then I know where it is stored, and I'm in charge of looking after it.

Possible Solution - A Shadow Server

Here is an idea that would address the issue of data reliability and access during a server/internet outage: a shadow server. This would be a local server within a business that maintained its own copy of all data on the SaaS server, and provide access during any external outages. It would also act as a redundant copy of the data should a Sidekick type data loss occur.

Google Gears would be another alternative, but while it works well for things such as Google Docs that operates on single files, it probably wouldn't work well for applications with large databases; you are unlikely to want to cache a company's entire database on every machine, especially using an SQLite database (which is what Gears uses).

The few times that I suggested something like a shadow server to an SaaS company (I can't remember to whom), they never replied. Allowing their server software to run on a customer's machine isn't something that most of them wish to contemplate, given piracy issues, and the many different server configurations that customers could have. However, it could be run on a virtual machine, allowing them to use a standardized installation, to administer it remotely, and lock down the server as much as possible as an anti-piracy measure. Virtual private servers have been around for a while now, so the technology is there. Not everything could be run on the shadow server, e.g., PayPal would be unwise to allow a shadow server to perform transactions, but having full access to their data locally would generally put customer's minds at ease, and make people less reluctant to use SaaS.

There is one major technical issue for a shadow server, data synchronization. Let us say that a company's data link goes down, but the main server is still running. The CEO makes changes which are stored in the shadow server, and so does the sales manager, who is currently remotely accessing the main server whilst on a business trip. When the data link is restored, the servers would have to be resynchronized, and any conflicts (e.g., two people edit the same item) resolved.

I know that at least one person is going to say "ah, but your power supply failure would take down your local network, and cut you off from the shadow server." True, until I plug my machine into the server directly, or into another hub (assuming that I have multiple units). Moreover, I can get up and do something about it instead of waiting and hoping for someone somewhere to do it and, that, is more satisfying.

Final Comments

Everything in life has an element of risk, so the issues above are no reason to reject SaaS. I personally use a few of them myself. However, I do hope that SaaS vendors will consider allowing customers to have a shadow server, or provide some other form of offline capability that is functional; allowing customers to save their data to a text file, PDF, or a report, is not good enough. Cloud computing may sound cool, but there is something reassuring knowing that your data is on a disk next to you, rather than on a disk (set of disks) somewhere on this planet.

I'd be interested to know what others think of these issues, including the opinions of SaaS companies. Feel free to comment below.