Steve's picture

Amazon starts HPC node offerings in AWS

Amazon announced this morning that they're offering a new node type in EC2 - HPC nodes! Yea, say goodbye to your huge linux cluster that you paid millions of dollars for, now you can get it on-demand from Amazon and spool-up with a click of a button! :) This should put some more companies out of business pretty quickly. SGI? Linux Networx? Aspen Systems?

Basically, the node type is the same, but the nodes are guaranteed to all be high-CPU, co-located and have some special low-latency network cards in them.

http://aws.amazon.com/ec2/hpc-applications/?ref_=pe_2170_16291170

Steve's picture

Downsizing and consolidating my computer equipment

Anyone who knows me knows that I've got a rack full of equipment in my basement. Well, most of the gear in my rack is getting a little old and it's time to clean house and re-purpose some of that gear. My rack takes about 12 Amps to power and I think that cutting down on some of the gear will help with the electricity bill too. I also got some money recently, and without spending too much money, I'm going to see what I can come up with for a new setup.

I'll be replacing:

* Dual 1-Ghz PC
* 7-drive SCSI RAID5
* Desktop PC (just the PC part, not the monitor or anything else)
* A couple of older Cisco switches

With:

* A Drobo with the DroboShare controller
* A laptop (probably i7 w/6GB ram)

I'll also try to move my older desktop linux machine into the old rack to save on desktop space. This should cut down on about 50% of my power usage, and the Drobo should provide a better backup and storage platform than my old SCSI RAID that is functional, but basically a relic of the 90's.

I'll re-post when the re-org has been completed.

Steve's picture

My Mechanical Turk Experiment

So, for about two years now, I’ve been using Amazon’s Mechanical Turk service to generate a little extra income.  It’s turned out pretty ok overall.  I can do it during ‘idle’ times and not miss out too much on what’s going on around me.  If I had a better laptop, I could do it while watching TV, but my laptop sucks and is tethered to a wall socket and network connection.  I managed to generate about $2000 between May and November last year which was enough to cover most of our family’s Christmas expenses.  Here are some things that I learned along the way.

I found some hits (jobs) that were fairly well-paying and were semi-reliable and paid enough to be worth my time.  $0.10 per job that took a few seconds to complete was my target for the baseline job type.  I found a good job and requestor (employer) on turk that paid out regularly, but unfortunately, a bunch of other ‘turkers’ also liked this job, so it became difficult at times to get work because others would snatch up the jobs faster than I could hit the refresh button.

So, I checked the terms of service for Mechanical Turk.  In short, the TOS says that you can do whatever you want as long as a human does the work (not computer-solved) and you don’t mess with or break their site.

So, I went about writing a script to pull in jobs on a regular basis – I was sure that others were doing this too, so I figured that it wasn’t a bad thing to do.  There is a queue of 10 open jobs in the Turk system that a ‘turker’ can claim and then work on at their leisure.  My script polls for work and if it finds a job, it pulls it into my queue so I can work on it.  Then, I’ve got another script that returns completed work.  It’s all done in perl and seems to work pretty well.

Steve's picture

Looking for work in 2010 – part 2

JobSearchNewspaper This will be the last part in this series because I offered a position after 20 days of looking for work and I don’t need to continue the job search anymore.  The job market is good, but there are a *ton* of people out there looking for work, so it’s not easy.  I think that I shaved and dressed up more in the last 20 days than I’ve done in the last 60 days of working at my old job.  :)  I’ll try to post some hints for anyone else in the job market.

Steve's picture

Getting laid off was the best thing that could've happened to me

100_5579Normally, getting laid off is not a very happy thing. However, when I was laid off from Pronto this summer, it was probably the best thing that could've happened to me at that time. I got to spend time with my kids for 3 weeks. Normally, I would only have an hour in the evenings after I got home in-between dinner and bedtime, but I've really had a chance to bond with my children this summer. Swimming, going to karate lessons, parties, hanging out in the back yard, running through sprinklers, ... It's these times that kids remember the most (my kids are 2 & 6 years old) and I am very thankful to be able to be part of those memories. I also love going to large, outdoor public swimming pools and we've done that a lot now that I've had some time off from work.

I had some time to spend keeping the lawn short and watered in the hot summer days. I never have time to do this at all, but since I had a few weeks at home, I decided that I'd be a little more of a homemaker. I also messed around with Rosegarden, my synthesizers, XEN, ZFS and took a BBQ class - awesome!

I've had some time to re-evaluate my goals and dreams. Life changes - whether you initiate them yourself or they're thrust upon you, are excellent times to change direction. I've made some interesting decisions about how I live my life and do my work in the last couple of weeks and we'll see if they end up providing some much-needed relief down the road.

Steve's picture

ZFS log server

Messing around with ZFS on opensolaris a bit before Oracle kills it once and for all.

ZFS and opensolaris make for a fantastic log server. Logs compress well. ZFS has built-in compression, snapshotting, and the Sun/Solaris NFS server is leaps-and-bounds above linux as far as stability goes. Here's how to make a ZFS nfs server that uses transparent compression. It's super-simple.

Install opensolaris, log in as root and do this:

format

(that will give you the disks that it finds - pick out a few that you'd like to use for the ZFS filesystem)

zpool create vault raidz disk1 disk2 disk3 ...
zfs set compression=on vault
zfs set sharenfs=rw=@10.0.0.0/24 vault

Now, wasn't that simple?

Now, you've got an exported filesystem that has transparent compression, it's raided. Let's enable snapshots so we don't have to worry about file rotation!

svcadm enable svc:/system/filesystem/zfs/auto-snapshot:daily

It'll take a snapshot immediately for every new share, check it out:

root@opensolaris:/vault# zfs list -r -t snapshot | grep vault
vault@zfs-auto-snap:daily-2010-06-28-14:22 0 - 1.48M -

Delete some files and go get them again via the snapshot.

root@opensolaris:/vault# ls
mpdscribble.log syslog syslog.0 syslog.1.gz syslog.2.gz syslog.3.gz syslog.4.gz syslog.5.gz syslog.6.gz user.log.0
root@opensolaris:/vault# rm syslog
root@opensolaris:/vault# ls /vault/.zfs/snapshot/zfs-auto-snap:daily-2010-06-28-14:22
mpdscribble.log syslog syslog.0 syslog.1.gz syslog.2.gz syslog.3.gz syslog.4.gz syslog.5.gz syslog.6.gz user.log.0
root@opensolaris:/vault# ls -la /vault/.zfs/snapshot/zfs-auto-snap:daily-2010-06-28-14:22/syslog
-rw-r----- 1 nobody nobody 394494 2010-06-28 14:00 /vault/.zfs/snapshot/zfs-auto-snap:daily-2010-06-28-14:22/syslog

Steve's picture

Looking for work in 2010 – part1

jobs

June 10, 2010, I was laid off from Pronto.com after 4.5 years of employment, by a guy that I interviewed for my own supervisor’s position.  Two hours after laying me off, the company immediately put my job description on craigslist in an entry-level capacity.  I loved Pronto – I had worked there since it was just a few months old.  I considered it “my baby”.  June 10th was not a good day for me.  Three days later they announced the buy-out of the rest of the original Pronto.com shares by their parent company IAC.

I’ve got a house, a wife to support, two kids and two broken cars that I need to fix.  Everything nowadays requires money.  I can’t wait to find work.  Every day that I’m unemployed, I lose $250.  Time to start my search for a new job, and fast!

Steve's picture

Mordac: Time To Upgrade

90190.strip.print

Syndicate content