• Storage Networking World, Day One

    So far the convention is going well. The morning was spent getting registered (painless) and then sitting in on lectures for HPC storage.

    Lecture 1: Next Generation File Services

    Not really a lot was learned in the first lecture,  as it seemed to be more of an introduction to storage for the HPC arena.

    Lecture 2: Accelerating Applications with Infiniband Connected Servers

    The slides for this lecture were good, and I will have to download them. There are a number of economic considerations for Infiniband that were mentioned in this lecture.

    Lecture 3: Solid State Drives

    Solid state drives appear to be gaining strength in use, but I don’t see them being used in everything for a couple of years, at least. It is a very young technology that needs some time to mature. There is still a wide variation in quality of the product, both in access capability (particularly write capability) and in number of writes for a given block on the media.

    Lecture 4: Fiber Channel Technologies: Current and Future

    About the only thing I really got out of this lecture was that 8gb/sec FC is on its way, and should be available for general consumption in the next quarter or two. In addition to this, it appears that 8gb/sec FC will actually be a lower cost point than 4gb/sec, which is useful information particularly if at work we decide to get new storage in about a year.

    Lecture 5: IP Storage Protocols: iSCSI

    About the only thing that I really took from this was that iSCSI packets are not aligned with IP Packets, which means that IP is responsible for breaking up the iSCSI packets for transmission.

    Lecture 6: Comparing Server I/O Consolidation Solutions: iSCSI, Infiniband, and FCoE

    This was a useful lecture, though I am now tossing up in the air whether or not I should have attended the lecture on AMD Bridge Bay technology. Anyway, there were some very interesting comparisons, with 10GBE apparently using between 4-8W of power for each port as compared to significantly less with other interfaces. A new version of 10GBE using copper Twinax connections does look somewhat promising, however. Limitations on this technology appear to be about 10m lengths.

    I need to pull these slides as well, as there are some references to things like CBA that looked really cool. Apparently Cisco has been able to do some booting over IB as well, though they did not go into exactly HOW they managed to do this.

    Lecture 7: SAS & SATA Combine to Change the Storage Market

    Another good lecture. Apparently SAS and SATA drives are interchangeable when it comes to the physical interface. SATA drives can be plugged into SAS backplanes without any trouble, but SAS drives cannot be plugged into SATA backplanes unless the SATA backplane has SAS capability.

    SAS also allows for aggregation, which means that it is possible to get 1120MB/sec on an X4 link. PCI-E x8 connections are capable of 1600MB/sec. This looks to be a very nice alternative to FC connections. Apparently it takes between 14-19 SATA 3.5″ drives to saturate a 4X SAS link.

    There is also a trend at the moment to move to 2.5″ drives in the server market. While this will increase spindle count and reduce power and space requirements, it sacrifices storage space.

    A tool that was mentioned and that is something that I am going to have to take a look at is something called SQLIO, which measures IO performance with real SQL queries against a filesystem.

    That is about it for day one. Hopefully tomorrow there will be more to write about.

  • More project work

    Just got over a major hurdle on my kd-tree project. It was segfaulting and I had no idea why it was doing so. Now I know, and that is one of the biggest hurdles in figuring out why something is going wrong. The fix is usually much easier to deal with once you know what is going wrong.

    This project was going well until I hit this snag… at which point I became disillusioned with it and stopped working on it for a good five days now. Finally I can work on it again with the knowledge that the problems that I run into are going to be small things, not big whoppers like this one.

    In other news I am going to be heading to Storage Networking World 2008 down in Orlando next month. It should be quite interesting to go to, as the subject matter is of interest to me. There is going to be some work to be completed prior to heading down there, such as gathering some facts of our work at the HPC Center to be able to talk about.

  • BARF

    Went to the Bay Area Renaissance Festival today with Athena and Ian. There we met up with some old friends like Amanda and Sarah. One other person that I knew there was Ashley, whom I have met at least one time before somewhere… probably at Hoggetowne.

    Anyway, BARF is pretty much the same as it always has been… a busier version of Hoggetowne but really something that is the same. The shows are pretty much the same, the stores are most certainly selling the same crap, and the food is the same over priced stuff that is available at every ren/med fair I have ever been to… except they were missing deep fried artichoke hearts, which I adore.

    There were really only two performances of note that I want to relate here. First was Christoph the Insulter, who bases his show entirely on insulting the crap out of crowd selected participants. For money, he insults them. His insults run towards the crude level of things, but there is some intelligence hidden in some of his words that take people some time to get at times. Not really my thing, but I respect him for his work, I guess.

    The other was the Chess board that is run at BARF. Now, I have been doing the Chess board at Hoggetowne for the past six years, and I can pick out a good fight from a bad fight on a board pretty darned quick. These were not fights that were all that good, I am sorry to say. Typically their fights run too long, with lots of gaps installed for speaking parts, which unfortunately don’t work all that well in an open air, full circle arena with a crowd that is screaming its’ lungs out. If you can’t hear what is being said, it isn’t really worth saying. This is why our board fights at Hoggetowne last perhaps thirty seconds, except perhaps the final fight on the boards… and even then we are finding that those need to be cut down somewhat. People just get bored with the fights, truth be told.

    Anyway, that is what I did today, in addition to a bit more work on the project for class that I have been assigned. Things are moving along on that, and tomorrow I actually get a chance to talk to the instructor about it and find out some things that need to be clarified.

  • Assignments and stuff

    I finally got my programming assignment for my databases class yesterday. We have been wondering in class when this was going to show up, and what it would pertain to. Now we know… and I have a feeling that there is a good portion of the class that is going to be hopelessly screwed.

    Why? Because this is a REAL programming assignment. That’s why.

    You see, these days in college level courses, at least at the University of Florida, we have been getting things like “make a program that takes these numbers and gives the average, mean, and standard deviation.” Of course, all of these programs are written in Java.

    This is different. With this assignment we are doing an exploration of search algorithms based on different indexing techniques. The first, our baseline, is simply a sequential search. I am still mulling over how exactly I am going to do it, as there are a couple of tweaks that I can perform even with a sequential search to get the best performance. After that we are to perform a kd-tree search on the data (the data is two-dimensional points, so a d-tree type indexing makes sense in this case) and then a Vkd-tree, which I have yet to actually figure out the specifics of.

    We have a month to do this assignment, and I have already started. Let me be the first to say that those that wait on working on this assignment are going to be screwed, because I am seeing issues with it already. I have sent three emails to the professor asking questions about the assignment (not for help, mind you, but about the specification) because certain things about it are not completely clear to me yet, such as the range on the points, whether or not they can be real numbers or if they are strictly integers, and just how many points there can be in a database file in the first place.

    All of these things matter, because if I write the program to use short integers and then the point values exceed 216 we get a buffer overflow. Same goes for if I want to store all of the points in memory, as I have a finite amount of space and need to define the number of points I am going to load. The sample database has 2 million points in it, which blew up a normal integer array size but survived in a short integer array. If the number of points is going to be larger than this, then I will have to go back to stuffing everything onto disk instead. Ah well.

    I have, however, decided that the output of all three methodologies are going to be output to disk first, sorted, then displayed. In this way I can get the same output and ordering regardless of search methodology, and I shouldn’t have to worry about sorting the original data first.

  • Fried an Indy

    Looks like last night I fried one of my SGI Indy’s, and it was the good one too!  🙁

    I have both an R4400 and an R4600 Indy, and the R4600 decided to fry itself last night. Not quite sure what is wrong with it, but when I turn it on, I smell ozone. I am hoping that the processor is not gone or something like that, so that I can just replace something like the power supply from the 4400 and be done with it. This machine had all kinds of interesting stuff installed on it too, like an nice hot video card (well, for its day anyway) and plenty of RAM.

    Ah well, this is what I get for playing with hardware that is 10+ years old I guess.

  • Donkey Kong Development

    Just read a great story about the development of Donkey Kong for the Atari 2600. It can be found here:


    I have read a lot about the history of computing over the past fifty years or so, and the attitudes and environment that he is talking about jive with everything that I have read so far about that time in software development.

  • netrek on gentoo

    So, I am writing my first ebuild file for gentoo, and of course I decide to do it with a package that has broken configure scripts (or at least they are broken on the Fulong… I am now working on playing with it on a standard x86 architecture to see if it is truly broken or if it is something that is broken only on the Fulong.)

    If it is broken completely, I guess I will be sending off an email to the developer of the Cow release of Netrek, and then writing a patch so that it installs properly. By default I am going to be throwing it into /usr/games/* because that is where it should properly go.

    *…time passes…*

    Looks like it is broken, but I went ahead and at least created a patch that will fix the problem. Of course, gentoo whines about all of the warnings that pop up during compile, but that appears to be more from poor coding of the software than anything else. I get the warnings in both x86 and mips versions. It is looking like it may be a long road ahead for me to get this cleaned up.

  • RIP Gary Gygax

    So Gary apparently died earlier today. 🙁

    I still have my set of First Edition books, and there is no way that I would ever part with them. I have a couple of 2nd and 3rd edition books, and I got the 3.5ed set as well, but at the core I am still a 1st edition player and those are the rules that I know by heart. I will be curious to see what they do with the upcoming 4th edition rules, though they have nothing to do with Gary.

    I also have the Dangerous Journeys Mythus core rule book, which is a hard one to find because of the lawsuits.

    It was because of Gary’s work that I was able to while away the hours when I was a teenager instead of getting into trouble out by the railroad tracks like some of my friends. I have many memories of my gaming that I have to thank Gary for.

    A message from a friend of mine really needs to be posted here, as it has some interesting information about Gary as well:

    I wonder how many of the TG folks even know who he is. Even the ones who play D&D. I sent him a Christmas card every year and got one back this year saying he was doing better and missed the “old days”. For me he was not just the guy who, along with Dave, came up with D&D. He was an old friend / acquaintance who I didn’t keep up with nearly as much as I should have.

    Gary never was a big fan of healthy life styles so I’m not surprised he died youngish. Over weight and a BIG fan of junk food. I can remember him sitting at the end of the table in “The Dungeon” (a game store in Lake Geneva) when we were play testing D&D and his Blackmore campaign. He would have a few giant bags of Cheetos and or Fritos and a 12 pack of coke and by the end of the night they would all be gone and he would be bumming from the rest of us. He and Dave were great guys who really got the shaft by big business.

    He will be missed.

  • Speed testing of Fulong

    Reading Ryan Hill’s blog, he did a test to see the merge times needed for a bunch of KDE packages (why people want KDE in the first place is beyond me, but whatever…) Apparently he did these on an SGI O2, and I was curious to see how they compared to the Fulong. The following is my result:

    kiona ~ # for pkg in kdeaddons kdeadmin kdeartwork kdebase kdeedu kdegames kdegraphics kdelibs kdemultimedia kdenetwork kdepim kdetoys kdeutils kdewebdev; do genlop -t $pkg; done
    * kde-base/kdeaddons

    Sat Jan 12 13:12:16 2008 >>> kde-base/kdeaddons-3.5.8
    merge time: 55 minutes and 54 seconds.

    * kde-base/kdeadmin

    Sat Jan 12 01:31:28 2008 >>> kde-base/kdeadmin-3.5.8
    merge time: 18 minutes and 51 seconds.

    * kde-base/kdeartwork

    Fri Jan 11 15:24:05 2008 >>> kde-base/kdeartwork-3.5.8
    merge time: 22 minutes and 38 seconds.

    * kde-base/kdebase

    Fri Jan 11 01:36:48 2008 >>> kde-base/kdebase-3.5.8-r5
    merge time: 4 hours, 59 minutes and 46 seconds.

    Sat Mar 1 17:54:52 2008 >>> kde-base/kdebase-3.5.8-r6
    merge time: 4 hours, 59 minutes and 12 seconds.

    * kde-base/kdeedu

    Sat Jan 12 01:12:37 2008 >>> kde-base/kdeedu-3.5.8
    merge time: 1 hour, 52 minutes and 53 seconds.

    * kde-base/kdegames

    Fri Jan 11 14:37:26 2008 >>> kde-base/kdegames-3.5.8
    merge time: 1 hour and 55 seconds.

    * kde-base/kdegraphics

    Sat Jan 12 04:00:52 2008 >>> kde-base/kdegraphics-3.5.8-r2
    merge time: 2 hours, 29 minutes and 24 seconds.

    * kde-base/kdelibs

    Mon Jan 7 21:44:56 2008 >>> kde-base/kdelibs-3.5.8-r2
    merge time: 3 hours, 56 minutes and 15 seconds.

    Tue Jan 15 12:47:09 2008 >>> kde-base/kdelibs-3.5.8-r2
    merge time: 4 hours, 22 minutes and 31 seconds.

    Fri Feb 22 14:16:42 2008 >>> kde-base/kdelibs-3.5.8-r3
    merge time: 4 hours, 17 minutes and 16 seconds.

    Sat Mar 1 04:58:21 2008 >>> kde-base/kdelibs-3.5.8-r3
    merge time: 4 hours, 17 minutes and 41 seconds.

    * kde-base/kdemultimedia

    Sat Jan 12 05:28:55 2008 >>> kde-base/kdemultimedia-3.5.8-r1
    merge time: 48 minutes and 45 seconds.

    * kde-base/kdenetwork

    Fri Jan 11 23:19:43 2008 >>> kde-base/kdenetwork-3.5.8
    merge time: 3 hours, 25 minutes and 1 second.

    * kde-base/kdepim

    Sat Jan 12 12:16:22 2008 >>> kde-base/kdepim-3.5.8
    merge time: 5 hours, 51 minutes and 46 seconds.

    * kde-base/kdetoys

    Tue Jan 8 02:13:33 2008 >>> kde-base/kdetoys-3.5.8
    merge time: 13 minutes and 19 seconds.

    * kde-base/kdeutils

    Fri Jan 11 18:14:59 2008 >>> kde-base/kdeutils-3.5.8-r1
    merge time: 1 hour, 7 minutes and 58 seconds.

    * kde-base/kdewebdev

    Thu Jan 10 18:10:38 2008 >>> kde-base/kdewebdev-3.5.8
    merge time: 1 hour, 28 minutes and 11 seconds.

    I estimate that I am getting anywhere from 10-100% better performance out the Fulong than he is out of his O2. If I get the Origin 200 back up and running, I’ll see what happens with that as well. Probably won’t bother with the Indy, however, as the numbers would just be stupidly long.

  • Fulong issues

    So, while I have been doing a compile of compiz on the Fulong, it wanted to compile kdebase as well. Fine, no big deal, except that it fails the compile of that package about halfway through. Upon restart of the compile it completes without any issues. This is a problem that I have seen (and reported) before with the Fulong, and now I think I have the answer as was so graciously bestowed upon me by Thiemo Seufer:

    The problem is a compound of
    1) Not enough RAM (only 512 MB) in some machines, which causes an
       increasing number of package builds to use swap, and some of them
       to evenutually fail to build because of a timeout.
    2) Slow on-board PIO IDE, from which the firmware can boot from
    3) A kernel-imposed limit of 1 GB when PCI DMA devices (like a SATA
       disk controller) is used.
    4) A kernel bug in the cache coherency management which hits PIO IDE,
       and causes instability since kernel 2.6.18. Up to then, the problem
       was mostly papered over by an excessive amount of cache flushing in
       the kernel code. This problem went unnoticed upstream since PIO IDE
       is these days only used on very small/cheap systems, where a
       different code path is used.

    So, the first one is the most telling, at least for my situation. I suppose the other problems (at least 2 and 4… 3 doesn’t apply since I don’t (and cannot)  have SATA controllers on the Fulong at this time) are also applicable, but I haven’t really noticed them one way or another.

    Thiemo notes that problem 3 has supposedly been fixed in kernel 2.6.22+, and since I am running at the moment, this should be fixed for me as well, not that it matters. He also has a fix in for 4, and it is waiting for upstream review. 2? Well, his solution is to add SATA disks, so apparently it is not really a problem for me.

    His answer to 1 is to add memory to the system, and recommends 1-2gb. Well, seeing as I already have 1gb in the system, obviously this is still an issue regardless of what he recommends, and hopefully a real solution will come about.