Windows Home Server Saves The Day
posted on 01/04/10 at 09:49:59 pm by Joel Ross
On November 11th, The Wife's laptop needed to be rebooted. But it didn't turn out to be a normal reboot. Why? Because it wouldn't complete that reboot for nearly two months.
It hung while rebooting. It wouldn't repair the OS install. It wouldn't come up in safe mode. It just wouldn't come up at all. Oh, it would get half way. But it wouldn't ever finish. After running a few diagnostic tests, it turns out the hard drive was bad. It was under warranty, but that's where things got.interesting. We bought it at Circuit City before they went out of business. It's a long drawn out story which I'll leave out, but I will say this: Circuit City warranties are serviced by the same company that Radio Shack uses. As a result, I'll never buy anything that's covered by a warranty from The Shack. It was an all around horrible experience.
But, we did finally get it back - on January 3rd. I wasn't worried about any data on the drive, and it came back with a fresh Vista install (which is odd, since it originally came with XP!). I booted it up long enough to see that it worked, and then immediately took it into The Dungeon. I shut it down, threw in the WHS recovery CD, and booted it up. 3 minutes later, the restore was running.
Most people say it takes 15-20 minutes to restore an image. I didn't see that, and I was connected via a 1 Gigabit connection. It took a little over 5 hours total. And you know what? I'm OK with that. Sure, 15 minutes would have been nice, but frankly, it's still a TON better than loading all of the software by hand and then restoring all of the data manually.
This is the second time that my home server has saved me - the first time was just days after getting it, and I got a virus on one of my machines. It was going to be painful to get it off, so I just restored the latest backup. But this is the first hard drive failure that I've experienced since getting my WHS. I knew in theory it should be this easy, but I'm glad to see it actually work (you know, other than the whole "losing a hard drive and waiting 2 months to get it fixed" part).
And more importantly, The Wife is extremely happy. Talk about a WAF builder!
Tags: WHS | Windows Home Server | backup
Trying Out Personal Kanban (Or At Least My Interpretation Of It)
posted on 11/09/09 at 11:02:31 pm by Joel Ross
One of the biggest things I struggle with is a way to manage what I need to do. I do a pretty good job of that at work, but in my personal life, things tend to get dropped. And at work, I feel that the amount of overhead I spend tracking what I need to do is too cumbersome. I've tried many different methods to track this stuff over time: Tasks in Outlook. Tasks in Gmail. To do list applications. Remember The Milk. A Todo.txt file. A personal Wiki. Evernote. One Note. Email. I even tried Paper once (that didn't end well. I can't read my own handwriting!). I'm sure there are others too.
How did that work out? Well, none stuck, so now I have tasks and notes about tasks all over my hard drive, notebooks, and spread throughout the Internet. Not exactly ideal, that's for sure. And because I tried new methods all the time, finding details about those items was tough, because I had to remember what the tool of the month (week!?) was when I was working on that particular item.
About a month ago, I saw a tweet from Joey Beninghove about PersonalKanban.com. That prompted me to read just about everything on that site. And it really got me thinking about how I could use this. I considered using an actual board, but ultimately decided to go with an online solution, since I wanted to be able to use this for more than just what I'm doing while I'm in The Dungeon.
The best online solution? Zen! It's a great Kanban solution built by Nate Kohari (author of Ninject) and his wife, Nicole. They've done an awesome job with the site. Adding tasks is about as frictionless as you can get, and moving them around is as simple as dragging them around.
The first step for me was to determine what stages my work goes through. In a typical project, these would be things like Backlog, Needs Defining, Ready to Build, Building, Testing, Waiting Deployment, etc. But for a personal board (at least for me), it felt more natural to define my stages as rough timeframes. After thinking about it for a while, I eventually settled on a set of "swim lanes".
- Far: This is my backlog. I could have left it as back log, but I felt Far fit better with my naming. This is where the whole universe of everything I need to do is stored. If I'm talking to my wife about something I need to do next Spring, I put it in here. If my boss says I should do something in a couple of months, I throw it in here. This way, it's tracked, and I don't have to remember it. It basically holds tasks that are more than a month away or I haven't had a chance to prioritize. I'll talk about how tasks get moved out of here later.
- Near: This is for tasks that I expect to start working on within the next month. By the time the task gets to this point, it should be fairly well defined. Any last details needed before work can begin should be figured out, and once it's ready, it gets marked as such. If it can't be defined, or it's waiting for another task to complete, I'll mark it as blocked so I know not to pull it forward.
- Here: Tasks in this lane are ones that could be done at any time - I expect that most tasks in this queue will be worked on in the next week. They should be defined and ready. If they aren't, there should be a reason - like it's due soon or it's blocked by something that's currently being worked on.
- Now: This is what I'm currently working on. It's fully ready, shouldn't be blocked, and just needs my focus to knock it out.
- Done: Once a task is done, it gets moved here, freeing me up to move onto another task.
- Archive: I have different criteria for how a task moves from Done to Archive, but basically, once a task is confirmed done, it gets archived.
So those are the steps my work goes through. It was nice to define that, but I didn't realize one of the biggest values of a Kanban board until I got everything entered and started using it for a little while. This screenshot illustrates it fairly well.
Can you see it? It has nothing to do with what the actual tasks are either. It's the visualization of my work. I use colors to tell me how tasks are categorized: orange ones are work tasks. Yellow are blogging activities. Teal are for Develomatic tasks, and blue are personal ones. I can, with a quick glance, get a good feel for what type of activity I need to focus my time on. I could never get that with any other task management applications.
So how does work go from one swim lane to another? It's a little different depending on what the task is for. Personal tasks are a bit different than work tasks, and Develomatic tasks are different than blogging activities, but the process is similar. I make a point to review the board on Sunday night, and get ready for the next week. This works out well, since at TrackAbout, our iterations are Monday through Friday. So tasks that I complete during the week get moved to Done, and on Sunday night (when we branch and update our staging site), I move the tasks completed in the last iteration to my archive. Personal tasks get Archived whenever I feel the job is done (read: The Wife tells me it's done!). Develomatic tasks get archived once it's been tested / deployed.
Any left over tasks that I'm still working on stay in the Now column. Since I don't usually have time for major personal tasks during the week, a lot times they'll get moved back to Here, or if I know I won't get to them the following weekend, back to Near. Any work in Here that I know I will do on Monday will get moved into Now, and any work that needs to get done the next week will get pulled from Near to Here. At that point, I check Far and look for things that are ready to be pulled out. If there's planning that needs to go into a task that I know will be coming up, I might create a task to handle the planning and drop that in Here. Basically, by the end of Sunday night, I like to have a rough plan for what my week will look like. Work tasks are usually the most up in the air because I don't totally control the priorities, but it's usually pretty stable for 2-3 days worth of work.
This review process happens a lot of nights, but Sunday is when I do a more thorough review and look into my backlog. Most of the other times it's simple a matter of moving more work into Now if I know I need more because something was completed that day.
For anyone familiar with Kanban, you'll notice one thing lacking on my board: WIP limits. That's work-in-progress limits, for those not familiar with Kanban, and the idea is that, in order to focus on the tasks at hand, you limit the amount of work you're doing at any stage, and you don't pull something else into that stage until there's a free spot. I haven't found a good balance here, and I think that's because of mixing different types of tasks. If I set a WIP limit of 2 for Now, and have two work tasks going, I have no room for anything else. While that may be just fine with my boss, I still have other commitments in other areas. So either I'm constantly moving tasks in and out based on what type of task I'm trying to do, or I do without hard limits. Moving them around all the time means more administration. Too much friction from the tool, and history tells me I'll eventually stop using it. So far, it's been a couple months and I haven't missed a day without checking my board. Ok, I may have missed a Saturday here or there, but that's about it. And I'd say I haven't been dropping tasks like I used to, which is the main goal of all of this, right? I do have soft limits in my mind. I try to limit work tasks to 2, personal tasks to 2 or 3, blogging to 1 and Develomatic tasks to 2. Although, even that's not a hard limit because a lot of times I try to batch up a bunch of small unrelated tasks and just knock them out.
Anyway, that's how I've been doing Personal Kanban. Doing something similar, but better? Let me know in the comments!
Full disclosure: As a member of The Lounge, I received a free Personal membership to Zen. Having said that, everything I'm doing for my personal kanban board can be done with the free account.
Category: Personal
Encrypting and Decrypting Data Using X.509 Certificates
posted on 11/02/09 at 08:24:08 pm by Joel Ross
Over the past few days, I've been working to change our build process and make it a little more flexible for our needs. As part of it, I was asked to look into how we encrypt our data, and see if we could come up with a better way to do it. Right now, we use DPAPI to encrypt information. It works great, but because the encryption is specific to a given machine, it means that encrypted values on one machine can't be decrypted on another machine. If we ever moved to a web farm, we'd be in trouble.
So I started looking at different options for how we could do it. The recurring solution I saw pointed to RSA encryption using X.509 certificates. Since you have the option to export your private keys for these certificates, you can share them across machines and decrypt across a web farm. And all the other side needs is the public key to encrypt the values.
I'm not going to into details about how to get the certificate or export the keys, but there are great resources for creating a test certificate (follow steps 1-3) and for importing and exporting them. These are the two links I used to get my machine ready. Note that in a production environment, you probably want to look at getting a certificate from a trusted certificate authority.
So now you've got your certificate set up, and you want to encrypt something. Now what? Well, don't do what I did my first time implementing it! I went along happily encrypting and decrypting passwords and everything was great. I even rolled the changes out to our team. Then I realized that there's a limit to how much text you can encrypt using this method. Apparently, for a 1024 bit certificate, it's 117 characters. Basically, the formula is (number of bits) / 8 - 11. So a 512 bit key will encrypt up to 53 characters. I don't know enough about encryption to adequately explain why this is, but a few searches turn this up over and over, and my own testing proves this out.
So what to do now? We have strings that can be over 117 characters. After talking it over with a co-worker, I realized that the correct way to go about this is to actually use a symmetric encryption, like AES. But that presents a problem, because both sides need to know the encryption key - a secret that everyone knows! And if we were to use the same key over and over, the encryption could eventually be compromised. So we need to change up the key every time. But how will the other side know the key? That's where the certificate encryption comes in. We encrypt the key and send both the encrypted key and the message to the recipient. The other side grabs the encrypted key, decrypts it using the certificate, and then can decrypt the actual message. From my understanding, this is how SSL works, as well as PGP.
Theory's good, but code is better, right? So how do we do this? First, let's look at how AES encryption works. There's a class in System.Security.Cryptography called RijndaelManaged that implements AES for us. I created a class that wraps the complexity of using it. The constructor takes in an instance of the RijndaelManaged class, and uses that to both encrypt and decrypt data. First, the encryption method.
1: public string Encrypt(string plainText)
2: {
3: MemoryStream msEncrypt = null;
4: CryptoStream csEncrypt = null;
5: StreamWriter swEncrypt = null;
6:
7: try
8: {
9: var encryptor = _aesAlgorithm.CreateEncryptor(_aesAlgorithm.Key, _aesAlgorithm.IV);
10:
11: msEncrypt = new MemoryStream();
12: csEncrypt = new CryptoStream(msEncrypt, encryptor, CryptoStreamMode.Write);
13: swEncrypt = new StreamWriter(csEncrypt);
14:
15: swEncrypt.Write(plainText);
16: }
17: finally
18: {
19: if (swEncrypt != null)
20: swEncrypt.Close();
21: if (csEncrypt != null)
22: csEncrypt.Close();
23: if (msEncrypt != null)
24: msEncrypt.Close();
25:
26: if (_aesAlgorithm != null)
27: _aesAlgorithm.Clear();
28: }
29:
30: return Convert.ToBase64String(msEncrypt.ToArray());
31: }
The encryption uses the instance filed _aesAlgorithm to perform the encryption, which actually produces a byte array. I then convert that to a string, since that's what I want.
Decrypting data is essentially the same process, but reversed.
1: public string Decrypt(string cipherText)
2: {
3: MemoryStream msDecrypt = null;
4: CryptoStream csDecrypt = null;
5: StreamReader srDecrypt = null;
6:
7: string plaintext;
8:
9: try
10: {
11: ICryptoTransform decryptor = _aesAlgorithm.CreateDecryptor(_aesAlgorithm.Key, _aesAlgorithm.IV);
12:
13: msDecrypt = new MemoryStream(Convert.FromBase64String(cipherText));
14: csDecrypt = new CryptoStream(msDecrypt, decryptor, CryptoStreamMode.Read);
15: srDecrypt = new StreamReader(csDecrypt);
16:
17: plaintext = srDecrypt.ReadToEnd();
18: }
19: finally
20: {
21: if (srDecrypt != null)
22: srDecrypt.Close();
23: if (csDecrypt != null)
24: csDecrypt.Close();
25: if (msDecrypt != null)
26: msDecrypt.Close();
27:
28: if (_aesAlgorithm != null)
29: _aesAlgorithm.Clear();
30: }
31:
32: return plaintext;
33: }
Now, let's look at the certificate side of things. I did the same thing here too - a class that wraps it and make things easier. But instead of having an algorithm for how the encryption should work (like RijndaelManaged is), this class takes an identifier for the certificate. I used the Thumbprint from it, since that was already in place in other areas for our system, and was a unique value I could grab. So, for this class, I pass in just the thumb print.
Encrypting is pretty simple with a certificate.
1: public string Encrypt(string plainText)
2: {
3: var signatureCertificate = GetCertificate();
4: var rsaProvider = (RSACryptoServiceProvider)signatureCertificate.PublicKey.Key;
5: return Convert.ToBase64String(rsaProvider.Encrypt(plainText, false));
6: }
The complications to the process reside in the GetCertificate() method. Once you have the certificate, encrypting is straight forward. Note that you only need the public key to encrypt - so you can pass around your certificate without the private key and they can encrypt data, but not decrypt it. Only holders of the private key will be able to decrypt it.
Decrypting is just as simple.
1: public string Decrypt(string cipherText)
2: {
3: var signatureCertificate = GetCertificate();
4: var rsaProvider = (RSACryptoServiceProvider)signatureCertificate.PrivateKey;
5: return Encoding.ASCII.GetString(rsaProvider.Decrypt(Convert.FromBase64String(cipherText), false));
6: }
Note here that it's using the private key from the certificate to do the decryption.
The heart of using this method is in actually getting the certificate. To do that, we have a method that retrieves it by it's thumbprint.
1: private X509Certificate2 GetCertificate()
2: {
3: var x509Store = new X509Store(StoreLocation.LocalMachine);
4: x509Store.Open(OpenFlags.ReadOnly);
5:
6: foreach (var storeCertificate in x509Store.Certificates)
7: {
8: if (_thumbprint.ToLower() == storeCertificate.Thumbprint.ToLower())
9: {
10: return storeCertificate;
11: break;
12: }
13: }
14:
15: return null;
16: }
By the way, the actual code is a little more, umm, robust for our purposes, but I eliminated the noise in favor of clarity.
We have the pieces in place for this to work, and now we just need to pull it all together. Again, I made a class that wraps the whole process and makes it easier to use.
First, the encryption part:
1: public string Encrypt(string plainText)
2: {
3: var aesAlgorithm = new RijndaelManaged();
4: var aesDataProtector = new AesDataProtector(aesAlgorithm);
5:
6: var rsaDataProtected = new RsaDataProtector(_thumbprint);
7:
8: var keyCipher = Convert.ToBase64String(rsaDataProtector.Encrypt(Convert.ToBase64String(aesAlgorithm.Key)));
9: var ivCipher = Convert.ToBase64String(rsaDataProtector.Encrypt(Convert.ToBase64String(aesAlgorithm.IV)));
10: var textCipher = aesDataProtector.Encrypt(plainText);
11:
12: return keyCipher + ivCipher + textCipher;
13: }
There's a lot going on here. But here's the gist of it. Get an AES encryption implementation, give it a key and initialization vector (IV. Honestly, I'm not totally clear on how that's used, but it's required and needed later to decrypt the string), encrypt those values using the certificate, encrypt the message with AES, then send them all along for the ride.
Decrypting is similar, except we have to do things in a certain order. First, we have to split the message up into it's three parts. Then we can decrypt the key and IV using the certificate, create our AES implementation using those values, and from there, we can decrypt the message.
1: public string Decrypt(string cipherText)
2: {
3: var rsaDataProtector = new RsaDataProtector(_thumbprint);
4:
5: var keyEnd = 172;
6: var ivEnd = 344;
7:
8: var keyString = cipherText.Substring(0, keyEnd);
9: var ivString = cipherText.Substring(keyEnd, ivEnd - keyEnd);
10: var keyBytes = Convert.FromBase64String(keyString);
11: var ivBytes = Convert.FromBase64String(ivString);
12:
13: var aesAlgorith = new RijndaelManaged
14: {
15: Key = rsaDataProtector.Decrypt(keyBytes),
16: IV = rsaDataProtector.Decrypt(ivBytes)
17: };
18:
19: var aesDataProtector = new AesDataProtector(aesAlgorith);
20: return aesDataProtector.Decrypt(cipherText.Substring(ivEnd, cipherText.Length - ivEnd));
21: }
Most of this is basic string manipulation. Again, for simplicity, the keyEnd and ivEnd values are hardcoded (and work with a 1024 bit certificate), but in my app, they are determined by the location of terminator values. I'll leave that part as an exercise for the reader. But note that encrypted values are typically terminated by an equals sign.
And that's pretty much it. The nice thing about this is that the only thing both sides need to know are details about the certificate. And if someone just wants to send you an encrypted value, you give them your public key, and that's it. No exchange of secrets at all!
Category: Development, C#
TrackAbout Is Hiring!
posted on 10/18/09 at 10:39:08 pm by Joel Ross
It's been almost a year since I started at TrackAbout, and so far, it's been one of the best decisions I've made. I absolutely love working from home, and the team I work with is extremely talented. We're a small company, but looking to grow the team very soon. We're going to add a couple of developers, and we're specifically looking for an experienced developer with compact framework experience. We're doing a lot of work in that area, and getting someone with knowledge there would be hugely beneficial. We're also seeking a QA lead, but unlike the development positions (which are both work from home), this would be in Pittsburgh.
There isn't a job posting for these anywhere, and I think part of that is because we want to see what kind of people we can find on our own through our blogs or twitter. Last year, there was a job posting, but both of us that joined TrackAbout found out about the position through blog posts (well, technically, I found it on twitter, but it was a link to Joey's blog). I think we'll find better and more qualified candidates this way, so I'm excited to see who responds.
If you're interested, please let me know. I know a lot don't necessarily want it to be public knowledge that they may be interested in another job, so feel free to email me rather than comment (jross at rosscode dot com), or check out my contact page for more options. I'd be glad to have a chat via IM, Skype or phone.
For a bit more information, check out this guest post by my boss about how we communicate on a daily basis and some of the tools we use. Also, here's part of the blurb from the last job posting TrackAbout posted.
Company:
TrackAbout, Inc.
www.trackabout.comTrackAbout is a small, profitable and growing software company that provides Software as a Service (SaaS) asset tracking services through the combination of a hosted web platform and various mobile devices that run our custom software. We have a solid, international customer base, are experiencing high demand for our services and are looking at exciting times ahead. We are looking for an exceptional and experienced developer who is interested in playing a pivotal role in the growth of a small company. If you're truly passionate about technology, we want to talk to you.
Location:
TrackAbout's main office is just outside of Pittsburgh, PA. However, our development staff is completely virtual (no office) and each developer works from home.All members of our development team are dedicated to keeping up with the latest techniques, technologies and tools related to software development. We take time out to share our knowledge with one another and improve both our skills and our distributed development environment. We often experiment with and integrate new tools into our environment. We use ReSharper, CruiseControl.NET, TDD with NUnit, FXCop, NCover, Bugzilla, and Drupal to supplement our development efforts.
Category: ASP.NET
Essential Versus Accidental Tasks
posted on 10/13/09 at 08:00:00 pm by Joel Ross
I finished reading Mythical Man Month a while ago. [Actually, it was a LONG while ago, but this post has been in my draft list for well over a year!] Overall, I thought it was a good book, but not as good as I expected. I think part of it was the amount of praise the book receives, and part was how good the first few chapters were - they left an expectation for the rest of the book, and it just didn't seem to live up.
Anyway, one of the chapters (No Silver Bullets, I think) talks about whether software development techniques can get an order of magnitude increase in efficiency, and lays out development tasks in two categories:
- Essential Tasks: The tasks that are directly related to building the software. Basically, writing code.
- Accidental Tasks: The tasks that you have to do to build software, but aren't technically part of the software.
The essay goes on to talk about how accidental tasks would have to be 90% of development time in order to get an order of magnitude reduction development time if you could remove them all, and how that's unreasonable. As a result, you have to attack the essence of software development if you want to achieve those types of results.
It's a logical argument. And while eliminating (or greatly reducing) the accidental tasks won't yield an order of magnitude speed up in development, it doesn't take a huge effort to reduce the accidental tasks either. Before you can determine if it's worth reducing the accidental tasks though, you need to know how much time they actually take. If they only take 5% of your time, any effort towards reduction will seem wasteful with respect to the return. But if the accidental tasks start to approach 1/4 to 1/2 of your time, then reducing that will yield much larger returns.
Before we can answer how much time accidental tasks take, we should probably define what the accidental tasks are. Some of these may seem controversial, but remember, accidental tasks are anything that's not essential to the production software.
- The build process: On a lot of projects, this can be a major time sink. I've seen projects that do a weekly integration cycle and it can take one person a few hours every week just to get the software built and deployed. Even automated builds can be a time sink, both with the time to set up and maintain the build, but also the ongoing build tax - think "Check-in dance."
- Environment setup: Creating a development environment takes time. This includes setting up source control, getting the right people access, installing the necessary software and tools, setting up a QA/Test environment, etc. All of the "infrastructure"-type tasks.
- Testing: This'll probably be the most controversial task on the list. But the reality is that manual testing takes time. Writing automated tests takes time, and while they have great value in creating correct code (and solid design), the tests themselves aren't production code. Plainly, tests aren't deployed.
- Project Administration: Status meetings, time tracking, billing, code reviews, etc. Important pieces, to be sure. But not essential to the software.
- Bug Fixes: This one's a bit different, because this code actually does make it into production. But I'm including it here because bug fixes aren't essential to the advancement of new features - they're flaws in features already completed. Had it been done correctly to start with, this time wouldn't be necessary.
I'm sure there are other components I'm missing, but those would be the major "accidental" tasks I see. The first two are basically setup costs. They are essentially a "one time fee". And they're roughly the same for any new projects, big or small. That causes a bit of a conundrum. On large or long running projects, the savings (percentage-wise) is small, so the incentive to figure out a way to reduce it is small. On smaller projects, the savings (percentage-wise) is bigger, and so is the incentive to automate it away. What isn't there is the extra time needed to figure out how to automate those tasks away. If you rarely start new projects, then reducing setup costs isn't a big deal.
So, that really leaves the last three as ways to get significant gains by reducing accidental tasks. They're all repeating tasks, and could easily account for 25% of a project team's time, and that doesn't include the time lost to context switching. Moving to a leaner project management approach will help reduce project administration, but for most organizations, this is a fixed item without a lot of flexibility. If you're consulting, you need to track hours and project status so you can get paid. You can optimize it, but most likely, it's been optimized if the company has been around for any significant time.
So that leaves testing and bug fixes. Ironically, increasing one will reduce the other. The more time you spend testing, the less bugs you'll have. But manual testing takes a lot of time, so the most efficient use of a team's time is looking for ways to automate the testing process. Things like unit tests, integration tests, functional tests, UI tests - anything that can be automatically run in a repeatable fashion - this is where you can gain some real benefits. The payoff: you'll spend a lot less time fixing bugs, which means your team is moving forward, not looking back. Also of note, investing time into automated testing will make your software easier and safer to modify over time, and will actually reduce the time it takes to do the essential tasks.
I took the long way to get here, but the bottom line is this: Increasing one aspect of an accidental task will decrease other aspects of accidental tasks, as well as streamlining essential tasks. Sounds good to me!
Category: Development, Software
Dynamically Loading Config Files
posted on 09/08/09 at 12:52:01 am by Joel Ross
In my last post, I talked about dynamically loading and unloading DLLs. It works well, but I quickly ran into an issue: configuration data. The main application, which is what houses the DLL that's loaded dynamically, has a lot of configuration data, and I need that to do the processing I need to do.
Since one of the main requirements for the secondary application was that it had to have near 100% uptime, I couldn't include all of the main app's configuration in the secondary app's config - because it might get updated with new builds, which would require the secondary app to restart. So, I started looking at how you can load a configuration file dynamically. As usual, it's pretty straightforward:
1: var map = new ExeConfigurationFileMap {ExeConfigFilename = @"C:\Bins\app.config"};
2: ConfigurationManager.OpenMappedExeConfiguration(map, ConfigurationUserLevel.None);
Now, the DLL can load its own configuration information on the fly. For my application, I added a method to my BaseType abstract class:
1: public abstract void InitializeConfig(string configPath);
By implementing this method, I can now initialize the config from the secondary application after it loads the DLL, and all the secondary app has to know about is the location of the config file the main application uses.
Category: Development, C#
Changing Code In A Running Application
posted on 09/03/09 at 11:53:32 pm by Joel Ross
As part of a new application I'm building, we wanted it to work together with an existing application already in place - in fact, we wanted it to share the bulk of the implementation for saving certain types of data. The main requirement we had was that it would always be up. It could live through upgrades of the main application, and, at the same time, recognize changes to the main application, and use that new functionality when the main app came back online.
We considered a few different approaches. One was a file based approach. We could drop the data in a file, and the main application would pick it up and process it. But we wanted an interactive process - when data comes in, our main application might have messages that needed to be sent back to the caller. At least part of the process needed to be synchronous. Web services was another option, but that didn't solve our "always up" requirement during upgrades.
Eventually, I had the idea of dynamically loading the parts of our application that we need. As long as we kept the interface from this new application into the main application simple and flexible, we could get away without having to change the new application (that often), and just load in new versions of the main application. So I set about proving out the concept. As it turns out, it's very do-able, once you piece the right things together.
One of the key pieces is to have a shared project that both the main application and the new application know about. It'll be where you put your contract that the two apps will agree on. I did it with an abstract class:
1: public abstract class BaseType : MarshalByRefObject
2: {
3: public abstract string WriteSomething();
4: }
Since this was just a dummy application, my naming is a bit unrealistic, but I'm sure we can get past that. Now, we inherit from that in our main application, and, well, write something:
1: public class DynamicLoadedType : BaseType
2: {
3: public override string WriteSomething()
4: {
5: return "Version 1 was called.";
6: }
7: }
Simple enough. Now let's look at the other side. First, a quick note about application domains in .NET. You can't load a DLL into the current App Domain and still be able to unload it. It doesn't work that way, and people a lot smarter than I am can explain it better than I could. The bottom line is that you have to load the DLL into its own App Domain, and rather than unloading just the DLL, you unload the whole App Domain. Also, since we'll be loading a type from another App Domain, we're essentially using remoting, which if you paid close attention, would be why our BaseType inherits from MarshalByRefObject.
So, we create an App Domain, load our DLL into it and grab the type we want:
1: var setup = new AppDomainSetup
2: {
3: ShadowCopyFiles = "true",
4: ApplicationBase = @"C:\Bins"
5: };
6: _domain = System.AppDomain.CreateDomain("MainAppDomain", null, setup);
7: _baseType = (BaseType)_domain
8: .CreateInstanceFromAndUnwrap(@"C:\Bins\AppDomain.Dynamic.dll",
9: "AppDomain.Dynamic.DynamicLoadedType");
There's a lot going on here, and some of it deserves some explanation. First, most samples you find ignore creating your own AppDomainSetup class. I created my own because I wanted to use Shadow Copy. I didn't want to have the DLL that I loaded be locked. Now, I can drop in a new DLL over the old one, have my application recognize that, and automatically reload it.
Once the type is loaded, calling a method on it is as simple as calling any other method:
1: Console.WriteLine(_baseType.WriteSomething());
Using a FileSystemWatcher, I can watch the DLL I loaded, and when the DLL is changed, I can unload the App Domain:
1: _baseType = null;
2: System.AppDomain.Unload(_domain);
3: _domain = null;
Then, I load a new App Domain with the new DLL, and I'm back to processing calls - the whole time able to respond to incoming requests, even if it is to tell the caller that I can't handle the request and to try again later.
To prove this all out, I created a simple application that loops every second and if the DLL is loaded, it calls WriteSomething(). If it's not loaded, it just says so.
1: while(true)
2: {
3: if(manager.IsLoaded)
4: {
5: Console.WriteLine(manager.BaseType.WriteSomething());
6: }
7: else
8: {
9: Console.WriteLine("No File Loaded!");
10: }
11: Thread.Sleep(1000);
12: }
Also, when the DLL is loaded or unloaded, a message is written as well.
I started the application, changed DynamicLoadedType.WriteSomething() to return "Version 2 was called.", and dropped the updated DLL into the folder. Here's what I saw:
Without ever stopping the application, I was able to make code changes, drop them in a running application, and see those changes immediately. Not bad for an evening's work!
Category: Development, C#
Unit Testing Has Changed How I Write Software
posted on 08/18/09 at 01:11:26 am by Joel Ross
A few months back, I remember having a conversation with someone about my views on test driven development and unit testing in general. I told him that, while I understood the benefits, I hadn't been able to see any of them. As a result, I didn't do a lot with unit tests, unless it was an obvious case that didn't have a major impact on the rest of the code base.
Of course, the reality is the other way around. I never saw the benefits of unit testing because I wasn't disciplined enough to write the tests, despite understanding the benefits.
Looking back, it was a poor decision on my part. Once I took the time (read: it's the way things are done at TrackAbout) to actually do it, suddenly those theoretical benefits became real. But more importantly, I saw what happens when you lack those benefits. Older parts of our system aren't completely under test, and frankly, changing those parts of the code is both dangerous and time consuming.
I didn't realize how much my development style had changed until I started a new project recently. In the past, my style was to develop vertical slices of the application, starting with a UI, and building down from there. Why? Because without a UI, I had no way to verify functionality. But this time, I started in the middle, with the main service of the application. Or, to be more to the point, I started with the tests for the main application service. When I ran into a need for something outside of the scope of what the service should do, I created an interface for it, mocked out the interface, and developed against that. Once done, I picked an interface I'd created and started working on that one. I worked my way through all of them, until I hit an edge, such as database access, web service calls, or third party components. By the time I was done, I had the core functionality of the system written and tested.
And I hadn't written a single line of UI code.
The UI was fairly simple, and was pretty quick to create. But the amazing part was that I only ran into a couple of bugs in getting the application up and running. And they weren't bugs in functionality - they were issues with how I wired my components together (using Ninject of course!). Once i got that functionality working, I was off and running.
It was a new feeling for me. I knew it would work because I had proof that it did work. And when I needed to change something, I had a designated place to start making those changes (it's test class), and could ensure that it didn't have far reaching effects on the rest of the system.
I still have a ways to go. I find myself just kind of flowing as I start to write code and suddenly realize I have a whole section that isn't tested, and I have to go back and write some tests to cover that. I still struggle with testing at the edge - where I'm finally to the point of writing the code that integrates with another component - be it a web service, an actual physical device or just a third party component. I find myself leaving that to test through in the application. I also haven't figured out the best way to test the UI or the Javascript I've been writing either. But, even then, it's easier to find the issues with them because I know the stuff under it is already tested and verified.
I know I'm late to the party on this, but I prefer to think of it as fashionably late.
Category: ASP.NET
Acer Aspire EasyStore Home Server Review
posted on 06/25/09 at 11:07:33 pm by Joel Ross
Ever since Windows Home Server hit the streets, I've been eying them enviously. I looked at building my own, buying an OEM copy and repurposing a machine I had laying around the house (which was serving the same purpose, albeit all manually), and buying one of the HP models.
Ultimately, I delayed long enough for another option to be on the table: The Acer Aspire EasyStore Home Server. To me, there's two things I'm looking for in a home server: storage space and expandability. I don't really care about the extras that HP offers - the media capabilities and backing up Macs being the main ones. I'm pretty happy with my home network setup, other than my backup strategy.
Don't get me wrong. I had everything backed up - on a nightly basis. I used DeltaCopy on all of my machines, and scheduled it to run nightly. For my machines, I actively ensured that I'd selected accurate folders for backing up important data. For The Wife's laptop, I made sure that I backed up her important folders and told her where to put files to ensure they were backed up. I used Live Mesh to sync important files between machines and "The Cloud". And on a weekly (or so) basis, I used an external drive to do full backups of my main machines. Despite automating a lot of it, it was still a pretty manual process. I had to ensure I was backing up the right things, and if a drive did fail (luckily, it hasn't happened), I'd be stuck reloading the OS and programs manually.
Oh, and those weekly backups? Yeah, I might as well call them monthly.
Anyway, back to the home server. Acer (wisely, in my opinion) decided to keep it simple, and put out a home server without all the bells and whistles. It does what it says it will do (backup your machines reliably and simply), and does it at a great price - $399. So I got one. A few days later, it showed up on my doorstep.
For the record, that's a Steve Yzerman bobble head monitoring the unboxing in the upper left.
I couldn't resist opening it, and it turns out it's a nice looking machine. All of my pictures of the outside were blurry, but frankly, that works since I'm more interested in the inside. Hot swappable drive bays (another advantage over building my own or repurposing an existing machine) are very nice!
Setting up all of my machines to backup was a breeze. With about an hour of work, I had all of my machines set up. I have 5 machines backing up regularly now. The initial backups took a while - I forced a few to start, and each machine took between 2 and 5 hours, depending on how much data and its network connection (the slowest was our older desktop, which has about 100 GB of data and is connected via a wireless connection). Now that they're all backed up, the nightly backups are relatively quick. I set backups to run between midnight and 7:00 AM, and they usually finish before 3:00 AM. One other nice thing is the home server is capable of waking up machines to back them up. I'm able to set my machines to sleep or hibernate after an hour of inactivity, and safely rely on them waking up to get backed up. Before, I had to leave them on at night, so they'd get backed up (or schedule them to be backed up during "busy hours").
So far, I haven't done too much with it, other than using it as a backup server. I have loaded some videos on it, and can watch them on my Xbox 360 (who needs the media capabilities of the HPs anyway!). I installed PlayOn Server on it as well, so I can play Hulu videos on my 360 as well. Getting PlayOn installed is pretty simple, but requires installing Window Media Player 11 on the server first.
As I mentioned on Twitter, I've added my WHS box to Live Mesh, which required jumping through a couple of hoops, but nothing too major. And the end result was worth it - I now pull down mesh data into my personal share, which then gets duplicated on there - that means that any data I add to Mesh is in at least 4 places: The machine it was added on, the cloud, and on the home server - twice!
About the only other thing I've done is hook it to a UPS. I have the APC Back-UPS ES 550VA, and while it's not huge, the only thing I want is for the server to shut down properly if the power goes out. One note here: do not use the built-in software that comes with the UPS. Everything I read says it can result in data corruption. Luckily, there's Grid Junction, a nice, free add-in that can be used. It's simple to install and it picked up my UPS without issue.
I did get an extra drive to go in the server, so I have 2 TB of backup space to work with. With having 5 machines to back up, I honestly wasn't sure how much space I would need. If I look at my disk usage and just add it up, it's about a terabyte of space. After running daily backups for a little over a week, I've used about 400 GB. I'd heard it was pretty smart about how to avoid duplication, and I'm sure there's some compression going on, but that's pretty impressive. Since I have two drive, I also enabled folder duplication for all of my shared folders. There's a chance those shares could be the only place a file is located, so having it backed while on the server is just as important as if it's on my machine, right?
I really can't think of anything bad to say about the machine. It works as advertised, it's small, and it's quiet. If you're looking for a solid machine to manage backups, I would highly recommend it.
Category: General
What’s this? Ads In My Feed?
posted on 04/29/09 at 11:52:59 pm by Joel Ross
For those who read RossCode.com by visiting the site, this post won't mean much to you. For those who subscribe to the feed, you may have noticed a new item in the feed: An advertisement. And that's a big change for me.
About a week ago, I joined the RSS Room of The Lounge. It was not a decision I took lightly. My view on advertising has always been to make money on casual visitors (mostly from Google searches), but not to make money off of the dedicated readers of my blog - the ones who subscribe to the feed.
So putting an ad in my feed is a bit hypocritical then, right? Well, not exactly, and let me explain why. Until joining The Lounge, the ads on my site were never really relevant. Yeah, Google thought they were relevant, but I never looked at the ads and thought, "I'd use that." That's not the case with the ads on my site now. I look at the companies advertising here and realize that I use their software, and I feel like they actually add value now. So putting ads in my feed no longer feels like I am just making money off my readers. Your opinion may differ, and if it's a real problem for you, please let me know why. I'd love to get feedback.
Oh, and while we're at it, The Lounge is conducting a survey to get feedback on who their target audience is. You'll also have a chance to win all 41 of the Manning In Action books, so please take a moment to fill it out. I did, and it doesn't take more than a few minutes.
Category: Blogging
:: Next Page >>
