Subscribe to kjeldahl.net RSS

Some notes about node.js

Submitted by Marius Kjeldahl on Mon, 2010-07-26 22:15

I've been a perl programmer for a long time, and it has served me good. As computers and web browsers became faster, using Javascript for web pages and lately, web applications, became important. For some time I've been using Javascript when and whereever it made sense, but mostly for client side "sugar". Perl is a great language, has a great community with plenty of high quality libraries ("CPAN"). Most scripting languages are appreciated for their simplicity, and less for their speed. For common tasks they are typically quite speed, usually because of low-lever C libraries running beneath it. If you ever had to run some "inner loops" in most of them, you realize there's usually a limit to what makes sense to do in such high-level scripting languages. I've personally "prototyped" a backgammon position evaluator (identifying all legal moves), a poker hand evaluator (finding the best hand from X cards, like holdem, omaha, stud or whatever) and a high frequency trader bot, evaluating positions, exit and entries on a per-tick basis. The great thing about the scripting languages is that they allow you to spend more time on the core problem, and less time convincing the compiler about all kinds of details in your data and algorithms. For the examples I mentioned (and more), rewriting in C/C++, either parts of the whole part of the application usually makes sense. Most of the scripting languages interface quite easily with C/C++.

File synchronization across systems (Your personal "Dropbox")

Submitted by Marius Kjeldahl on Wed, 2010-07-21 17:46

After upgrading my development system and servers to newer linux releases (Ubuntu 10.04 in my case), it's that time again where I need to look at how I synchronize work across my machines. My work pattern is pretty standard; in my office I have a desktop computer with a multiscreen setup. In addition I have a laptop that I carry with me everywhere else. I also have a server which hosts my personal web server, email and other apps I'm working on that are server hosted.

I used to have a Windows laptop as well, but less and less work (from clients, and involving my own projects) touches Windows, so the requirement to synchronize with Windows is less of an issue than it used to be. In fact, Windows work so well running under linux virtualized now that it can be treated quite similar to other "embedded" type development systems, minimizing the need to actually work from within the Windows environment. When needed, I develop outside of Windows, where the Windows machine is set up to access files directly from the host.

Mac OSX is similar enough to other unix/linux system that most solutions should work pretty well on OSX, and I will not treat it separately in my notes below.

The solutions I will comment on below are:

  • Dropbox
  • rsync
  • Unison
  • Persy
  • git-sync.sh
  • Jake
  • git (updated 23:03)

At the end of this article I have also written a short summary of what I ended up choosing.

Dropbox

Unless you have lots of content or need to manage your own privacy/security issues, http://dropbox.com/ should solve most, if not all of your problems. They have client support for Windows, OSX and Linux, and possibly more. Everything is sync'ed to Dropbox' own servers. The only thing you need to do is sign up and install the software, and you can sync a few tens of gigabytes easily.

Moving from Kubuntu to Ubuntu

Submitted by Marius Kjeldahl on Tue, 2010-07-20 13:06

For the last few years, I've been running the KDE version of Ubuntu, named Kubuntu, on my desktop computers (laptop and desk computer). The last few years, KDE has progressed a lot, and with a couple of exceptions I have been very pleased with my KDE based desktops. One recurring problem with Kubuntu/KDE however has been it's wireless connection applet; the thingy that manages your wireless connections on a laptop. Another one is managing multiple screen, which is supported by the KDE GUI, but for some reason never remembered and had to be redone for each reboot. Alternatively, I could "fix it" my mucking with Xorg.conf, which I've done on numerous occasions. Not fun, but certainly possible. And don't get me started on KDE audio. ALSA or Pulseaudio? The various "layers" and legacy emulators to get audio working everywhere, including flash is a puzzle I would gladly leave to somebody else to figure out, and save me the trouble. The wireless problem has persisted for several years, and other stuff, like the multi-screen management thing, simply doesn't seem to be fixed. And I don't feel like fixing it myself.

Since I mostly by the bleeding edge laptop computes, I've gotten used to some amount of tweaking to get my laptop usable, typically meaning installing more modern kernel releases and some bleeding edge packages from unofficial software repositories, or even from source. Because of this I've also given my systems the benefit of the doubt, in that it is probably my ignorance that breaks something when something isn't behaving properly.

Server and site upgrades

Submitted by Marius Kjeldahl on Mon, 2010-07-19 14:19

I have recently upgraded the server serving these (and other) pages. It's been lots of fun. I've almost forgot how funny it is trying to piece together postfix, dovecot, amavisd, clamav and procmail to get proper mail handling, including virus and spam filtering and secure authentication. Sign me up for wanting new more modern email standards! Anyway, things are starting to fall back in place, and I hope to have most things running smoothly shortly.

By requests, I will also be republishing some of my older articles, including analysis, tools and details about reverse engineering the data format used by the AAII SIPRO (American Association of Individual Investors Stock Investor Professional) product.

Javascript, Dojo and long-running animations

Submitted by Marius Kjeldahl on Mon, 2008-09-01 00:00

I recently needed some simple animation effects for a project, basically gradually changing the background colour from "hot" to "normal" on certain topics. I have a backend that processes realtime information and logs it on a webpage which is streaming (using the perl-based Meteor server). What I wanted to do is that new log entries "glow" with a hot color, gradually changing to normal over time, typically 15 minutes in my case. Dojo has objects named animateProperty for doing this, which works great. There is a catch however. By default, those animations run at 100 frames/second, which is fine if all you're doing is animating a few items for typically less than a second. If you want to animate a page with 50 entries with animations lasting 15 minutes each, it is not so great. It works fine, but it eats a lot of unecessary cpu cycles.

Digging through the Dojo sourcecode, I found a simple solution. Here's an example to illustrate:

  anims[0] = dojo.animateProperty(
    {
      node: dojo.byId("line0"), duration: 60*1000*15,
      properties: {
        backgroundColor: { start: "#7f0000", end: "black" }
      }
    }
  ).play ();

Reading through the Dojo sourcecode (dojo.js.uncompressed.js), one of the first thing animateProperty does is to create an underlying dojo._Animation instance and passing all the arguments to it's constructor:

        var anim = new d._Animation(args);

Looking into the dojo._Animation definition, it has some instance variables as well. Of special interest is:

        // rate: Integer
        //    the time in milliseconds to wait before advancing to next frame 
        //    (used as a fps timer: rate/1000 = fps)
        rate: 10 /* 100 fps */,

So by simply modifying the rate when creating the animateProperty instance, we can control how often we want the animation to do actual work (the timeout of the timer advancing frames I guess). The solution then becomes:

2007 - bad year for income/dividend investing

Submitted by Marius Kjeldahl on Tue, 2008-01-15 00:00

With the success growth stocks had in 2007, it is easy to point to dividend/income stocks as under/non-performers. An article in Economist December 13th writes (http://economist.com/finance/displaystory.cfm?story_id=10286619):

Those funds that invest on the basis of dividend income have also suffered badly. Alex Stewart, a strategist at Dresdner Kleinwort, says 2007 has been an annus horribilis for income investing.

As always, it depends. Using the backtester I tested the common Silver_Parachute screen (a dividend screen using a 26 week momentum sort) invented by DrBob:

Define {Silver_Parachute}
Uses [SI Price] [SI Volume--Average Daily 10d] [SI Yield] [SI Price Change 26 week]
Deblank [SI Price] [SI Volume--Average Daily 10d] [SI Yield] [SI Price Change 26 week]
Keep :[SI Price]>5
Keep :[SI Volume--Average Daily 10d]>10
Keep :[SI Yield]>=10
Sort Descending [SI Price Change 26 week]
End

Holding the top 10 stocks for four weeks, averaged across the first four possible starting weeks in 2007 gives the average return of 35% (GSD 25). The worst of these four rounds gave CAGR/GSD of 23/21. Not bad at all. That's using Friday's closing prices for sells and buys, which isn't possible of course since the picks are actually generated the following Sunday using Friday's closing data. This only shows the returns from price appreciation, not the dividends themselves. DrBob's tests have shown average yields from the picks of this screen to average 10%-15% per year, which is in addition to the average 35% return quoted.

Migrating from Joomla 1.0.x/SMF 1.1; bridge sessions to Drupal 5.x sessions

Submitted by Marius Kjeldahl on Thu, 2007-11-01 22:23

Joomla has served me nicely for quite a while now. But over time I have become frustrated with the lack of "native" comments and forums and the various hacks to get comments/forums working nicely. I've had some success with SMF, but the pain one needs to endure to keep the user database, session cookies and passwords getting lost by themselves finally got to me. The one thing which I still believe is better are the Joomla templates supplied by Drupal templates still seems to be lacking. Thankfully, HTML & CSS isn't that hard anyway (not that I am claiming to be an expert), so with some external inspiration from other websites it is still possible to build something half-decent looking.

In addition to my homepage at kjeldahl.net, I also run a site dealing with investing and trading resources named keelix.com. On that site I actually hook into the CMS system's user authentication and session systems to allow people access to various resources. For simple web-stuff, I'm mostly a perl person, so hooking into foreign authentication and session systems is a bit more challenging than "including" some PHP snippets in PHP code/templates.

C# Windows development for unix/linux + emacs users

Submitted by Marius Kjeldahl on Fri, 2007-07-27 01:00

I have a background developing software for both Windows and Linux, although for the last 10 years I have been mostly doing development on and for Linux. Some time ago I took an assigment for a client involving developing Windows software. I made a first prototype using Perl on Windows which worked just fine, and the client wanted a C#/.Net version instead, so off I went.

It has been ages since I used Visual Studio, and it became clear that it was not an environment I would be particularily effective in anytime soon. Fortnunately, most of the development tools I have been using the last years are available for Windows, so off I went to see if I could set it up on Windows.

Editing 

Emacs runs j ust fine on Windows, so that was easy. Except the current version of Emacs for Windows does not ship with syntax highlighting for C#. There are a couple of ways to get hold of these. I chose the following method:

  1. Get hold of a more recent cc-mode.el package t han the one that ships with Emacs for Windows. The one that ships with it is too old for the C# package I will install next. You can get the updated cc-mode package from The cc-mode homepage.
  2. After install ing this, install C# for Emacs.

To get cc-mode and csharp-mode to load correctly and automatically, you probably need something like below in your emacs startup file (c:/_emacs):

Turtle Trading the Euro FX (EC)

Submitted by Marius Kjeldahl on Wed, 2006-04-26 00:00

I have been working on finding strategies that work for real-time trading through an ATS (Automated Trading System). I did some initial tests using MACD, before moving on to implementing one of the systems in "The Original Turtle Trading Rules".

Original turtles logo

While the pdf describing the system used to be available in the public domain and easily downloadable from the net, it seems people are now charging for it. You might find it freely download other places it if you Google for turtlerules.pdf . If can probably buy it (from here) if you can not find it.

Anyway, the original turtles system (AFAIK) used daily charts. I wanted to implement and test the system using shorter timeframes, and eventually I implemented and tested it for various periods from 3 to 30 minutes. 

For backtesting strategies I obtained real time data from anfutures.com (see Datasources) and off I went. The data consists of one minute candle data, showing the contract date, timestamp, open, high, low, close and volume for each contract. There is little data prior to 2003-03, so I do not consider any results and/or data before this valid, even though the historical data from anfutures extend back to 2000. The Euro FX (EC) contract expires four times a year, so each contract is trader for three months.

The system I implemented is basically the "system 2" as described in turtlerules.pdf (a "Channel Breakout system taught by Richard Donchian" using a 55-day breakout, where I used a 55-candle breakout instead with varying candle lengths). Since I have no plans on doing the backtesting or trading manually, I needed some additional rules to make things sane:

Contributions and banner ads

Submitted by Marius Kjeldahl on Sun, 2006-04-23 00:00

First of all, thanks to everybody that contributed money to this project. The money contributed allowed me to upgrade the hardware increasing the backtesting capability significantly, which should benefit all users here.

As an experiment I have decided to stop accepting contributions for now and test Google AdSense instead. If money trickles in I plan to use this to subscribe to additional data services, and more hardware upgrades when that becomes useful. 

Pages