A Sys Admin's Tale

More on Cassandra

2010-03-31T17:18:00.003-04:00

I've spent the past little while playing more with the Cassandra database or key store. I've learned a bunch of new things.

I've got Cassandra setup on two dedicated servers and a third server to act as the web server. After a little bit of initial configuration issues, I've got both servers up and running in their own cluster.

Want the simplest piece of advice to get it up and running? Read the README. Best guide that there is.

It's interesting, from what I've seen so far, there are no user accounts to be created for accessing the database. This made me wonder initially how to secure the database from other people accessing it and manipulating the data maliciously. But I believe I figured out how to do this.

The nice thing, is you can setup the interface that Cassandra binds on. This pretty much leaves two different methods for securing Cassandra:

1. Setup the web server to run on the same box as Cassandra. This way, you can tell Cassandra to bind to localhost or 127.0.0.1, and it would therefore ignore all other requests.

2. Setup Cassandra to be bound to it's actual network address, but have the box Cassandra (and the cluster) is on in a location on the network that is not accessible on the outside. This way, the web server on the inside can hit Cassandra, but nothing from the outside will be able to.

More to come, but I'm interested in seeing where this goes so far.

Cassandra Continued

2010-03-18T21:05:00.002-04:00

So, I've delved more into using Cassandra as a database and it's still been interesting in learning about how it works.

Unfortunately, there really isn't much in the way of a user community who can help out when there are any issues. Additionally, there is not a lot in the way of documentation for setting up Cassandra, using thrift, or anything like that.

Right now, Cassandra is setup in a VM with 1 gig of ram and 1 processor dedicated to it. I've tested inserting data into the datastore. We ran a loop that inserted 10,000 records into the database, and it took about 1 second to insert all the records.

The record included a key, username, timestamp, age, and another timestamp. All records were inserted with a while loop in php. Now, the issues seems to be with reading the data out. We've been attempting to make a php page that will read a specified number of records and print it out to the web page. At the moment, the read is failing at reading anything above 2000 records.

I've talked it over with another individual that is working on this project, and we believe it might be with the "hardware" given to the virtual machine. At the moment, it only has 1 gig of ram available to it, and after reading, it seems that 1 gig is the minimum amount of ram that is recommended when using Cassandra.

Unfortunately, I am rebuilding my raid array at the moment, and will not be able to test out running the VM on my personal computer. Once the array is done being rebuilt, I can start testing it out with higher amounts of memory, and any other resources, and can see if I can get the read going with that.

Looks like it might be a project for the weekend. If anyone else knows about Cassandra, or have tried setting it up, definitely hit me up with how you have your rig configured. I'd be interested in it.

Cassandra...

2010-03-13T20:47:00.002-05:00

Well, it was an interesting day yesterday.

Nearly all of the day was spent trying to setup an environment in a virtual machine to use/mess around with the Cassandra Database. It's an interesting concept that I'm having almost a little bit of trouble trying to get my head around in learning how to use it.

The devs of the Cassandra database don't even really want it to be called a database. It is supposed to basically be a flat file and it is also not a relational database. It was a ain to setup yesterday.

The documentation is not the best. In order to use the Cassandra database, you basically need to install Thrift to act as the API that connects to the db. I worked with a developer to install the Thrift API first, and then we went about trying to install the database.

After bringing everything over and starting up the database, it would run in the background fine. However, I could not get Thrift to connect to the db and provide a CLI to mess around with the db. It kept having java connection refused issues when trying to connect.

Googling the issue said to change the location that the db is listening from "localhost" to the IP address or fully qualified server domain name. Making that change, unfortunately did not fix the connection issue.

We ended up finding a guide for making php interact with the db. After following that guide, we were finally able to insert some dummy data and actually get the php page to pull and display information from the database.

One interesting thing of note was that every column has a name, value, but you also have to give it a timestamp. That was one of the most interesting things I saw when beginning to learn how it works.

Anyways, more coming soon.

URL Short

2010-03-10T14:36:00.002-05:00

To take a quick break from tracker talk,

The past few days I've been working on trying to develop a web application to shorten URLS.

Yes, I know tinyurl already exists, and I also have heard of the other url shortening app that's been released.

I took this project on because I want to learn php and basically just sharpen my development and logical thinking skills. It looks like This will just be a couple of php pages with a database in the back storing everything.

It's been interesting going from ASP to PHP. To be honest, things just seem to be more intuitive using PHP. The language seems just easier to use and more logical. For example, connection strings to databases in ASP can be long, big, and most people have to take some time to look at in order to get how it is put together. PHP? It's a simple function with the database host, username, and pass. That's it.

So far, I'm liking this.

Any good designers want to come up with a sweet UI for the Admin Interface?

Choosing the OS

2010-03-04T15:48:00.003-05:00

In most cases, I think you are going to find that you will be running a tracker on a Linux box versus a Windows box. Most likely it is cheaper, and will be the option that you'll end up picking.

The only thing that comes with this, is there can be a learning curve. We had the option of picking either a windows box, or between various linux distros. In the end, we chose Ubuntu. I had some experience with using Ubuntu, and it is known as being a very friendly Linux OS.

Ubuntu also has just tons of documentation, guides, tutorials, and forums that you can read up on. Going into this, I only had experience with messing around on an Ubuntu machine, but only the desktop version. I've never used the server version.

The biggest "shock" to me was, "Wow. It really is only command line". At first, this was the hardest part for me. I felt more comfortable doing some of the finer tweaking of the server when using a gui, and I didn't have that option. However, it really was a blessing in disguise.

All I have to say is just give it a shot. When you want to learn how to install an application, look it up. How to configure a service? Look it up. In the end, it really helped me more fully understand the Operating System, and make me feel much more comfortable operating everything from the command line. When I had to do things that I didn't know how to do (which was nearly everything) I just looked up and followed a guide.

While it might be a slow start, you'll gain some very valuable knowledge doing it this way. Unless you are pretty knowledgeable with another distro, I'd recommend using Ubuntu as your main OS.

Hosting

2010-03-02T08:49:00.002-05:00

One of the most important things that you will need when starting off with your own tracker, is finding the right host. If you pick the wrong one/a bad one, your going to be having some major headaches later.

The big thing you are probably going to want (at least) is a VPS (virtual private server). Not the same as having a dedicated server, but really it is the bare minimum. I would highly recommend against using just a web host. Any good web host will soon be able to see what it is that you are actually running on it, and you can expect them not to be happy about it. Also, some plain web hosts have automated scripts that will auto slow down processes or anything similar to that that you account might be using. This will directly impact the performance of your tracker.

Trust me, we know, we tried. In the end, I would recommend a VPS as a starting point.

I would also recommend finding a good backup host. Just start doing your research, and find where you think another place your tracker could be hosted. In case your current host ever pulls the plug, you can try to minimize the amount of downtime that you experience.

Some basic things to think about as well is, how will you pay the host? Personally, I try to give as little personally identifiable information as possible. I'll buy a pre-paid credit card and use that to make payment to the hosting company/person. Another worthy question, is if you receive a takedown notice, is the host immediately going to take you offline? Will they let you respond? You have to try to feel out these sort of things and make the best decision for yourself.

Experiences with a Tracker

2010-02-24T14:04:00.003-05:00

After doing a little bit of thinking on what might be a good idea, I've decided to offer a little bit of information on what it takes to run a bit torrent tracker.

This will probably take place within a few other different posts that might be completely different in content, but these posts will come. Maybe other people who are interested in learning about this will find some good information from my thoughts on this and can do something better.

I had found a similar guide/insights from a different tracker Sysop. The thoughts of that individual helped provide some insight into good ideas and bad ideas while running this.

Hopefully, you'll be able to pick up a thing or to. As they come, enjoy.

Recent PC Build

2010-02-16T13:48:00.002-05:00

Just also thought that I would post a recent build of the computer I just made (about 3 months ago). It's awesome, and it's a beast:

*i7 950 Processor
*GTX 285 Video Card
*3 One Terabyte Hard Drives in a RAID 5 array
*12 gigs DDR3 Triple Channel Memory
*Intel P6T Motherboard
*Bluray Burner
*1000 watt powersupply

This computer stomps on and spits out Left for Dead 2 and Crysis

Time for Heavy Raining?

2010-02-16T13:12:00.001-05:00

So, I am a little interested in the PS3's newest release, Heavy Rain. I really don't know a lot about thsi game. It looks as if it is designed to be more along a story telling type of game, vs a traditional video game.

The girlfriend of mine actually likes watching me play games, and loves it even more when the story of it pulls her into the game. She has actually asked me to go play some games so she can watch and we can get further into the story. I feel like this seems like a game she would really enjoy watching.

I have to get her into Uncharted and Uncharted 2 as well. Both great games.

Any suggestions on getting a girlfriend into gaming?

Sharing and Pointing to the SQL

2010-02-09T09:22:00.002-05:00

Yesterday was a fairly productive day for myself. Finally, with a fully updated version of Windows Server 2003, I was able to install SQL Server 2008 and SharePoint Server on the VM.

The SQL Server installation went perfect. All that was needed was to create an account (a non-admin account) that SQL Server could use for starting the database services on the server. Other than creating this, all features were selected for installation, and all went without a hitch.

SharePoint Server was then installed without an issue. One thing I recommend when installing SharePoint is to not do it in stand-alone mode (where it installs its own express version of SQL Server). I attempted this about a week ago and was continuously having issues with setting it up. It would always fail when SharePoint would attempt to create dummy data.

After avoiding this option, everything was installed fine. I am in the process of configuring everything on the SharePoint. At the moment, I have a very quick test site up, but I am trying to figure out if I can get it to operate over port 80 versus the random ports that are assigned to it (upon creation). Also, anyone have any good suggestions or their own tips and tricks for configuring SharePoint?

If so, definitely share.

The Hosts File

2010-02-05T09:09:00.002-05:00

Just a quick tip for all of you out there. The Windows Hosts file can be a good asset that helps to make life a little easier and quicker on you when using your computer/server.

There might be times in an application that you are prompted to input a network address for the application to connect to. One common example, Remote Desktop. Instead of having to type out the IP address every time, you can make your own mini DNS Server like file. This file would be.... the hosts file.

What happens is you need to open and edit the existing hosts file that Windows uses. Within the file, you can make a string equal to an IP address. For example, say you have a kitchen computer at the network address 192.168.1.130. Instead of having to type the full IP address to connect to it, you can edit the hosts file. Make "kitchen" equal 192.168.1.130. When you do this, then your set. Go to Remote Desktop, and when it asks for the network address, type "kitchen". Windows will automatically reference the hosts file, it will see that the IP address corresponding to "kitchen" is 192.168.1.130, and it will connect to that address.

As you can see, it is basically what a DNS server does, except its located and used only by your computer. Hope this helped out.

It's Just Degrading

2010-02-02T18:46:00.002-05:00

Today has been a long one. I was working at my desk, on a server in a distant land, and about to have a team meeting, and then, the text message comes. "Meet me in the server room" Hm... I haven't received one like this before. Then a second, "I just got a call from security".

With this, myself and two others are expedited to the server room. As we arrive, we just hear this noise going off inside. After taking apart the server, looks like our raid array has been degraded as one of our drives has failed.

The bad thing is, this server is so old, that we are afraid to try to rebuild. If we do, it's very possible that the whole server can just crash and then we might lose everything. So, our first thought is to make a backup, while we still can, of everything on the server. Although as we attempt to do so, we discover that the dvd drive is broken. It won't close.

I find a server that is the same model, and attempt to switch out dvd drives, only to discover that for the life of me, I cannot get this drive out of the server case. I call another admin to come take a look at it, and 20 minutes later, we're still unable to get it out.

I end up placing the second server right next to the degraded server, and run a SCSI cable from the degraded server, into the dvd-drive of the other server. Finally, it looks like we're able to back this up. But then we learn more...

This server is an extremely old server. It was donated by a team who no longer needed it, and we decided to turn it into a file server for everyone. However, this server is so old, it doesn't even support USB 2.0. To backup all of the information on this server would take atleast the entire night, and possibly some into the next day.

Our only other option is a network backup. We create a new share on another server. We then load Acronis, and tell it to prepare a backup of the entire system. We were successfully able to get it to see the newly created share, and we begin its dump.

Yup, the degraded server has only a 10/100 NIC Card. So, it won't be gigabit speed, but it is faster than USB. The full backup is going to take about 9 hours. Looks like it's going to be an early morning for me in the office.

VMWaring me out

2010-01-31T15:23:00.004-05:00

So today has been somewhat of a long day. The girlfriend has been over going through all of my things in preparation of us moving in together. And by going through my things, I mean, me begging to keep things so they aren't thrown out.

But in the meantime, I have been really having some issues with getting this VM of Windows Server 2003 setup. I'm using VMWare to run the guest VM. My host OS is Windows 7 Ultimate x64. Everything seems to be running fine, but for the life of me, I cannot get this VM to maintain a connection to the internet.

The way the networking is currently setup at my place, is upstairs in the house I am in, is where the cable modem is located. Currently (although only for 1 more month) I live in the basement. I have two routers in this house. One router is a standard WRT54G router broadcasting the internet. The router in the basement is flashed with DD-WRT acting as a repeater bridge. My desktop is connecting to the repeater bridge for internet. There is no DHCP server running on the repeater bridge, so any machine connecting to that needs to have a statically assigned IP address.

My host machine is able to connect to the internet fine. However, the VM just keeps losing connectivity after 5 minutes or so. It's currently setup as having a bridged connection, and this is where it is losing its connectivity. However, this is odd, whenever I setup the VM to have NAT networking, the VM does not have any connectivity at all.

This isn't making sense to me. I could understand that incoming connections would have issues and not connect to the VM, but all outgoing connections from the VM should go through. I've also tried setting up another laptop to connect wirelessly, and then share its connection with an ethernet cord, and that does not work either. All the same issues.

For the time being, looks like I will only be able to update this very slowly. Hopefully the new networking (where I have direct access to the network) environment when I move to my new place will help solve this issue. Any ideas on the current situation though? Let me know.

TCL Me Silly

2010-01-30T19:07:00.001-05:00

So today has been a debugging day for myself it seems. I help run a couple channels on IRC. Or really, the bots that I manage run it. On the bots, I have a couple of scripts running that help to make the channel management easier. All scripts are written in tcl. These scripts help manage FAQs, key word triggering, and managing ops, voiced users, etc.

Now anyone who is on IRC knows that there are many types of bots that can be and are used to help manage channels. These bots that are currently being used (and running these scripts) are eggdrops. The one script in question is for managing oped users, voiced users, and the actions that they can perform.

The way eggdrops work (if you don't know) is you can set flags for users. For example, if a user is an operator, you can give them the +o flag (attribute). If a user is voiced, you can give them the +v flag. One basic check that the script should be performing, is that opped users should be able to kick voiced users from a room, but not the other way around.

I'll give you one guess as to what I'm trying to debug.

So, since voiced users do somehow have the ability I'm trying to figure out why. I've also narrowed down that it seems to only be happening in this one specific channel, and I think I just discovered why.

When you set flags for the user in the bot, you can set them on two different levels. You can set a flag (like +o) on a per channel level (where the flag is only for that channel) or on a global level. So what's happening, is the script is doing a check on the user's (who is being kicked) flags. It also compares those flags to the flags of the "kickee".

In this channel specifically, I've set the flags for the operators at the channel level, not the global level, and I think that is why this issue is happening. When the script does the check on the flags, it is doing a check on the global flags for the user being kicked, not at the channel level. Since the opped users in here have their flags set at the channel level, not global level, the script isn't seeing the "o" flag. It then thinks it is not set, and then kicks and bans the user from the specified channel.

So, right now, it looks like my (hopefully temporary) work around is that I have to set the flags at the global level for the ops so that they do not get kicked. In the meantime, I have to try to figure out how to make the script also check on the channel level.

Here is the script that I am working on at the moment. If anyone happens to be great with tcl, or even better, a guru with eggdrop scripting, take a look at it and let me know if you know how to make it do a channel check.

In the Words of Bill Gates

2010-01-29T21:50:00.000-05:00

"Hello World"

I've been thinking about documenting everything that I do with technology. Regardless if people read this, it will be a good source for myself to reference when it comes to problems/issues/anything I've done in the past that I'm doing at the time. I think this could be a good resource for many people.

To give a little info on myself, I am currently a Systems Administrator in a Windows environment for a large company. I love my work. I love being hands on with technology. I couldn't do anything different.

I'm also a pretty big gamer. At the moment, I currently have a Xbox 360, Xbox, Playstation 3, Nintendo Wii, and Nintendo DS Lite. All have been hacked and modified in one way or another, except for the PS3. But hopefully, that will be coming soon (will be a future article).

I enjoy collaborating with people to work on projects. I'm active on IRC, forums, and many other mediums. I think the point of this blog will be to just talk about what I'm doing when it comes to hacking consoles, setting up servers, installing services, creating networks, and really, just anything technology related.

I'd love to hear any and all input, thoughts, and anything else you want to send my way. But for now, I'll call it a night. I'm looking forward to keeping this going.