Author Archive

Next, on YOTG.

November 15th, 2011

It has finally happened.  My lab has finally gotten the funds to upgrade our aging G5 servers with nice, new, shiny Mac Mini’s and a Promise Pegasus RAID.

Currently, the lab I run has 10 Mac Pro desktop’s all running into a 6 year old G5 Xserve and Apple RAID.  The RAID uses 14 IDE drives that are basically maxed out.  We have about 4TB of storage on 14 drives..this is very sad.  Our G5 servers are not upgradable anymore, and we have limitations on the types of things we can serve on them.  They have lasted us this long, but it’s time to finally phase them out.

In the next 2-3 weeks I’ll be replacing our two G5 servers and our RAID with two Mac Mini servers and the Promise Thunderbolt RAID.  The servers will give us huge boost in performance and the RAID will bump us to 12TB of storage.  This will not be an easy task as our current systems all run 10.5 and the new servers run 10.7.  I will also have to migrate all of the user accounts and data to the new system without losing anything. Instead of removing our old servers I will use them only as basic servers; MySQL, Apache, Xgrid controllers, etc.  I’m also going to use them as tertiary backups for our user accounts and servers (backing up the new machines and user accounts to the RAID once per week).

Over the next few posts I will attempt to document the migration.  I’ll start with initial setup then go to migrating data/accounts then end with the final phase out process.  I hope that the next few entries may help people who get into a similar situation as well as keep a record for myself on any problems I might face.

Steve Jobs: February 24, 1955 – October 5, 2011

October 5th, 2011

Thanks, Steve.

Imaging a Lab with DeployStudio

September 2nd, 2011

Imaging is a great thing..it really is.  When you have more than 2 computers, imaging becomes your best friend…and if you buy new machines or one of your older machines dies or gets messed up, it saves so much time.  I have a complete backup ready to deploy at all times for both Mac and Windows.

I image my lab once a year. This ensures that I have the latest updates for every machine for all programs, but it also cleans out the old stuff from the previous year that builds up over time.  Apple makes imaging very simple by installing a NetBoot feature on all of their machines and including a NetBoot server installed with MacOS X Server.  In the past I used Bombich NetRestore, a free AppleScript based program that helped make NetBoot image sets and helped with deploying them.  Mike Bombich stopped making NetRestore and suggested everyone to try DeployStudio for imaging..so I did.  I must say that DeployStudio is an amazing program especially for a free program.  It’s also very simple to get running and fairly robust. In this post I’m going to go over image creation, setup, and deployment with DeployStudio (DS) and also go over some issues I encountered and how I fixed them.

Creating the NetBoot Set

The first step to any Mac NetBoot is the NetBoot set.  What the set is is a basic image file that includes all the tools your computer will need to read the image, copy the image, and even run checks on your computer even if you’re not imaging.  It’s a very basic MacOS install that resides on the server.  DS creates these images for both PPC and Intel machines in the same set, so any Mac can boot from the same set.  After installing DS on your server you can open the DS control panel and begin setting up your system AND create your  NetBoot set.  I will not be going over server setup in this post, I may save that for a later time.

The DeployStudio Control Panel

When you open the control panel you should launch the assistant (you can also find it in /Applications/Utilities).  When the assistant opens you select “Create a DeployStudio NetBoot set and continue.  If you’re running the assistant on a computer other than a server you will see this:

DeployStudio DHCP Setup

If you plan on using a server to do the deploying, you can skip this, if not, you’ll have to setup a DHCP server.  This depends on your setup, for my case I can skip this.  The next step allows you to name your set; set the name and unique identifier to whatever you wish, (unless you have multiple NetBoot sets). When you click continue you will tell the set where the computer should log in and look for the images and workflows.

My settings...

more settings...

The settings above are MY settings, yours will be different.  The login and password for mine are supplied by the LDAP server.  The final step is the actual save location and creation of the image.  Pretty self explanatory. It takes about 5-10 minutes.

Completed NetBoot .nbi file

After image creating is successful you’ll have a nice .nbi file in your save location.  This file is basically an image file that contains the bootable images for PPC and Intel as well as the basic MacOS system with some basic utilities like Disk Utility, Terminal and Startup Disk.  It’s roughly 2.5 GB and it should be placed on your server in the NetBootSP0 folder (It’s located in [Volume]/Library/NetBoot/).  Inside the NetBootSP0 folder will be other folders which DS created during install, these contain various other things for DS and also house your images.  I will go over image creation next.  This is where we will be able to test to see if your NetBoot Server and set are both working.

Creating Images with DeployStudio

Creating the images is an extremely simple task once you know what settings you need.  I will explain the setup with my current settings but attempt to go over most of the other ones.

To start the process, boot your mac and hold the ‘N’ key down during power on, this will perform a network boot (REMEMBER: Your computers must all be on the same subnet, this is the only way to do this without messing with a lot of things!)  If your computer boots to the DS screen you will see the DS Runtime Window.

This window shows all of your available jobs in DS.  There are a few default jobs that come with DS, we’ll make our own later for deploying.  For now we’re gong to select “Create a master from a volume.”  Click the Play button at the top and you will come to the heart of the Image creation.

My Image Settings from a PowerPC computer

This window is probably the hardest window we’ve seen so far.  First thing is to choose which drive you will make an image of from the dropdown menu.  I’ll start with my MacOS partition.  After selecting the correct partition I name the image something like 2011_09_02_Intel_lab and leave other settings alone.  The keywords are not very important unless you have a lot of images. I usually select Compressed for the type because it saves space and it gives a much faster restoration.    Access group is what you would have set in your initial DS setup that I did not cover.

Format is what kind of image you are making.  Since I’m doing a MacOS install the Format will be HFS+.  I normally select “Auto Detect” but if you want to have HFS+ Journaled, Case-sensitive or both you may want to change it because it will always auto-detect HFS+ without journalising.

Once my settings are correct I click the Play button at the top and the image making process begins.  This will take a lot of time depending on the size of the image being created,  a 100+GB image will take roughly 2 hours (sometimes more, sometimes less, depends on the machine and network) and it will then compress the image (my images get compressed to about 75GB from 128GB…compression rocks!).

Masters in the NetBootSP0 Folder

After image creation you will see the .dmg file in your NetBootSP0/Masters/HFS folder.  (Note: I just found out that new versions of DeployStudio won’t show your images in DS Admin unless you have .hfs in filename before the .dmg, it will automatically add them during image creation, but if you have old images, just add the .hfs right before the .dmg extension).

You can use this same process to create NTFS, FAT, and EXT4 images.  Follow the same steps but make sure you leave the Format as “Auto-Detect.”  After creating a NTFS image it might take some time to show up in DS admin, this is because some server-side tasks may need to be done, it will show up when that is complete.  NTFS imaging requires a little more setup in DS admin beforehand…again, I will not be covering that in this post.

 

Making Workflows to Deploy Images

DeployStudio comes with an administration program where you can manage images, workflows, packages, scripts, and see progress of NetBooted computers.  You can also set up all of your computers in it before hand (names, network settings, licenses, etc) and set up automation for all of your systems so if you want a computer to automatically format and re-image when you NetBoot it, you can do that (please don’t think that’s a great idea…just saying).  To start setting up workflows you’ll need to open DS Admin, it’s located in /Applications/Utilities.  Enter your server credentials and you’re presented with the DS server information.

The window that opens first is the current (or previous) activities.  In this window you can watch and control the computers that are currently working in DS.  ou can also see what jobs they were doing, and how far along they are.  This screen is very helpful when you have DS running on many machines.

I am going to explain how to setup a dual-boot Mac workflow.  The default jobs are very helpful at getting you started, I’m going to start from scratch.  To create and edit workflows we’re going to select “Workflows” from the left sidebar and begin setting up our job.  Click the “+” button at the bottom and you will be presented with a new blank job.  Then click on the little “+” button next to “Drop tasks here.”

Creating a new workflow

The first thing to do is to drop the “Partition a disk” task from the left side to the drop space.  Then you should select  “Mac OS X + Windows” from the Apply layout template dropdown menu.  Resize the partitions to suit your needs, make sure your images will be able to fit on the partitions you make for your drive.  I normally do 75% Mac OS/25% Windows, I also normally Automate this process, your mileage my vary.

The next step is to drag the “Restore a disk image” job from the left and drop it after the partitioning job.  Your MacOS image should ALWAYS be first of else it will not work.  Select “Enter value…” from the Target volume section, then select the “MacOSX” option from the menu.  Set your Image to HFS and select the appropriate image from the menu (the one you created earlier).  Now, for the options you can read from the image below how to set those.  If you’re imaging Mac OS 10.7 Lion you should check “Restore system recovery partitions” but I don’t need this.

My HFS Settings

You may also notice Multicast settings, you can set this up if you’re brave, I don’t need it so it’s ignored.  Your HFS partition is complete, now on to Windows.

Drag the “Restore a disk image” job from the left and drop it after the first restoring task.  Select “Enter value…” from the Target volume section, then select the “WINDOWS” option from the menu.  Set your Image to NTFS and select an appropriate image from the menu again.  Settings for Windows is relatively the same as HFS with some exceptions;  you should check “Expand restored NTFS partition” and uncheck “Set as default startup volume” unless you want to have Windows as your default.  You’ll also notice that all of these tasks are automated, this is so you can boot the computer, select the job, and walk away without intervention.

DS NTFS Settings

Now, you can add more jobs to the workflow such as AD binding, or software updates, but this setup is the basic setup for a dual-boot deploy.  Now just rename the job by clicking the name in the top with the other jobs and rename it, you can also add a short description of the job.  Your workflow is now complete! Now it’s on to the easiest task…deployment!

Deployment

I say this is the easiest part because it really is.  If you have everything set up properly, you should have no issues.

To deploy the image to the computers, boot the machines again pressing the ‘N’ key, when the machine boots to DS you can select the newly created Workflow and press the play button.  If you automated everything, that’s it..it will partition your drive and load the images to those partitions.  After the job is complete your computers will either tell you it was successful (or failed…more on that below) or they will reboot.  If the task was successful, GREAT!  Reboot the machines, they will run the final scripts in MacOS then reboot again…MacOS is done.  You only have one more thing to do and that’s configure Windows.  I won’t go into this because it’s going to be different for everyone, but you will have to activate windows and any other programs that require it because Windows will not keep the activation after imaging.

Issues?

Now, not everyone will be so luck to have a successful run…if you run into any issues visit the DS forums, they are very helpful and pretty speedy.  I had one issue that just drove me nuts.  When I ran my deployment script the MacOS partition would go fine but once Windows hit it would fail…everytime.  DeployStudio keeps logs for every computer on the server, so I took a look and noticed the following errors:

[Thu Sep  1 14:41:15] dyld: unknown required load command 0×80000022
[Thu Sep  1 14:41:16] -> invalid starting block value () defined in MBR for partition /dev/disk0s3.
[Thu Sep  1 14:41:16]    Check your partition map. You need to define at least one DOS/FAT partition in order to get the MBR automatically in sync with GPT.
[Thu Sep  1 14:41:20] -> Restore action completed.
[Thu Sep  1 14:41:20] Restoration failure (elapsed time: 0.24 minutes)

I posted in the DS forums (topic link) and in a matter of hours the admin of the forums posted a solution:

Sounds like the custom fdisk command fails on 10.7 DSS netboot sets. You may try to remove the one located in your netboot folder at /Applications/Utilities/DeployStudio\ Admin.app/Contents/Frameworks/DSCore.framework/Resources/Tools/fdisk.

So I tried this and BOOM, successful.  It’s great when a developer helps with products so quickly…and I’ve only usually seen this with free or open source projects.  So if you’re having issues, the forums are key.

I hope this post helps people out with Mac imaging and deployment.  If you have any other questions or issues feel free to ask in the comments.  If this post helped you or think it will help others please feel free to repost and share away!

A Long Overdue Thunderbolt Review-type-thing

June 23rd, 2011

Two months ago I said I would write something about the Thunderbolt..it was only supposed to be about two weeks but life happens I suppose.  Honestly, I wasn’t expecting to cave-in and buy the thing but I found a coupon for $50 off and thought, “well, this might be good!”  I bought the phone only a few weeks after I decided to root my original DROID (which from now on will be referred to as OG, Original Gangsta) and install cyanogenmod 7.  Being the happy owner of one of the best Android phones to ever come out I had high hopes for what was supposed to be Verizon’s new super flagship phone as well as the first phone to use their 4G LTE network.  What I got from it was a mixed set of feelings that make me miss my OG sometimes.

Let me get this out of the way first, I like this phone a lot.  It’s a great device, it’s fast, it’s sexy, it’s big..but it’s not perfect…far from it, and most of the reason is due to the crap software that is on it…let me explain.

The UI and Software

My OG ran stock vanilla Android (what devs call AOSP: Android Open Source Project). This basically means it’s the bare Android OS with no added UI tweaks and no added bloatware crap (with exception to some Verizon apps).  This is the best way to run Android for the most part because it’s not using the CPU or RAM to run some fancy/ugly user interface over top of it, it’s not going to have built-in apps syncing crap in the background, it’s just plain ol’ vanilla ice cream (and I like vanilla ice cream).  And it works!  It doesn’t waste CPU (meaning it’s faster) and it doesn’t background sync unnecessary apps constantly (meaning battery savings) unless you install them.  But of course, HTC and other companies want to change it to suit their own phones and needs.

Stock Android Screenshot from N1

This is a Screenshot from a Nexus 1 running AOSP 2.2

Now, the Thunderbolt (as with many HTC phones) doesn’t do AOSP Android, they use Sense UI, it’s their own user interface which they designed to put on mobile devices for a “sleek” and “unified” look.

HTC Sense UI Screenshot

This is Sense UI

While some people like the look (which I don’t think is terrible mind you) and the feel, I don’t like it, I hate it.  There are several reasons I hate it actually the first and most important being that it’s clunky and slow to respond VERY often; it feels like my OG before I rooted it…this shouldn’t happen on a 1Ghz phone with 700+MB or RAM.  So what I did to “fix” this was install ADW.Launcher and use that as my home app.  It’s faster, and it looks and feels more like a stock phone now.  Second, they package a whole mess of crap in with it; they have a “friendsteam” which gathers your Twitter and Facebook and whatnot into a widget on your screen and displays it and updates it for you; a weather widget (which is actually nice, but i should have a choice to remove it);  and their own Facebook and Twitter syncing built in.  This is stuff I’d like to install on my own and not have running in the background constantly asking me to log in and sync; if I want Twitter and FB, let me install them myself.  They also use their own MMS app which is very slow compared to the AOSP app.

 

You might be thinking, “man, this is cool that it’s all built-in, why are you hating on this?”  Because I like to be able to choose what to have installed on my device.  Because of these built in apps (which mind you, aren’t really apps, they’re more like utilities) I have two Facebook apps, and two Twitter apps and it’s kinda dumb to waste space on stuff like that.  I also find it annoying that they START UP WITH THE DEVICE…even when I DON’T USE THEM.  What a waste of CPU and battery.  Once CM7 is released for the phone (not Alpha-builds) that will be my savior.  Verizon is also guilty of bundling tons of bloatware with this device.  When I received the phone it was PACKED with tons and tons of useless crap..and you can’t remove this stuff!  I’ve written about carriers loading bloatware onto phones before, this shit needs to stop with Android.

The Battery

Smartphones are notorious for having crummy batteries.  My OG was actually pretty great before the last few months of using it.  I used to get over a day with it and I was happy.  When deciding on a new phone every review I read had the same complaint with regard to the Thunderbolt: The battery is terrible.  Now, I’m never far from a power source, and I have around 4 or 5 micro-USB chargers from my older devices and ones that I found around the lab, so this wasn’t too much of an issue for me.  My first day on the phone I got through the entire day of heavy use and texting without it dying, it got to about 5% by midnight (from 8am or so).  The second day, same thing.  On the third day that’s when it ended.  Without using it too much I made it to about 2pm before it started dying on me, I charged it and it was low again by 9pm.  Now I see what the complaints were about.  I tried using task killers to kill unwanted tasks, I synced data less, I lowered brightness, nothing helped.  I read around online and found the best way to get more mileage was to turn off 4G.  I figured, I don’t need 4G all the time, so it’s fine, if I get more battery time out of it this is what needs to be done.  After turning off 4G, I get through the day again.  A few weeks ago an update was released that helped the 4G radio consume less power, so since then I’ve actually been able to leave 4G on and get through most of my day with it…so good on them for fixing that.

The Hardware Itself

This is where the phone really does come out.  This phone has some weight to it (it’s heavier than my OG which was a brick) and it’s got that big bright screen.  The screen makes it so easy to read things on and to watch videos, the rubberized back makes it easy to hold, the weight makes it feel like you’re not going to crush it in your hands.  It is a nice, solid phone.  It’s also fast when you need it to be.  Playing games, running apps, downloading, it just runs fast (especially with ADW.Launcher).  Of course Verizon’s network helps too, I get 4G everywhere in NYC, and the 4G speeds are really good (EVEN INDOORS! Take that WiMax!).  I have plenty of space on the 32GB MicroSD card they give you and on the internal storage (8GB but only about 2.5 are available).  The only complaints I have about the hardware and design: no dedicated camera button, I really miss this, but it’s fairly minor; the bluetooth volume is super low, I’ve read that it only does it with some headsets but my Jabra is so low I cannot use it; and the GPS takes forever to lock on, sometimes 5-10 minutes.  Now, some of this is probably software-based so a fix may be in line (apparently there might be an update in the next month or so to address some of these issues as well as the random reboots caused by the last update) but until that fix is out these problems will remain.

Overall

So to sum this up in a few lines (TL;DR); I really like this phone a lot, it is a great phone by design.  The 1Ghz Snapdragon processor and the large amount of RAM really make this phone fly with apps and games and with Verizon’s 4G it really is a speedy phone in all faces.  However, the phone has many kinks due to some bad software that comes bundled with it and a broken update from HTC.  Most of it’s issues are solely based on the software so hopefully we will see fixes for them in the future.

 

That didn’t last as long as I was expecting…

April 18th, 2011

I decided to upgrade to the Thunderbolt after all.  Main reason: I got $50 off from a web coupon.  I’m going to write a little bit about it after I’ve used it a bit more.

Some first impressions: This thing is super fast, it’s got a nice feeling to it (not as solid as my DROID but still pretty well built), and the screen is really really nice.  The battery life isn’t as bad as everyone says it it…I got a full day out of it with fairly heavy use…we’ll see if that continues though.  I hate SenseUI…so I replaced it with ADW Launcher right away and I’m waiting for CyanogenMod to come out for it (it’s currently being worked on).  So expect some sort of review in the coming days/weeks…