May 16

Windows Backup

Windows Backup

Since the dumping of NTBackup, the semi-useful backup tool of the Windows stable for a good decade, many users have struggled to come to terms with its replacement Windows Server Backup (WSB).

If you have the time to get into it, you’ll find Windows Server Backup a far better tool and backup solution, with the caveat that it’s far less user friendly (and completely useless for tapes). However, the ability to “time machine” files, like a Mac, as well as offer a complete bare metal recovery restore method, is a pretty huge leap forward over NTBackup.

None the less, not every server has a spare hard disk to hold these copies and not everyone ones to plug in daily external hard disk, for a variety of reasons.

So instead, I was asked if Windows Server Backup can dump its data to a network share. The default answer is, no, WSB does not work over the network because it can’t maintain multiple versions, etc. You’ll find the WSB GUI simply makes no offer of a network location – only local ones.

However you can still actually backup to a network share – you just can’t do it with the GUI and you can’t maintain multiple verisons. For most people, this is probably ok and basically makes WSB similar to NTBackup, in terms of functionality. Many people used NTBackup to maintain network share based backups. If you have a large enough network share, you can maintain one folder for each day of the week, for example, giving the functionality of multiple backups.

So here’s the command line to backup the entire C: to a network share, \\server\share1\backup

WBADMIN START BACKUP -backupTarget:\\server\share1\backup -include:C:

This will create a folder called WindowsImageBackup on the share \\server\share1\backup which will hold a full duplicate of the C: of your server (and it’s better at collecting all files than NTBackup ever was, regardless of permissions). You can now run this form a batch file as a scheduled task and you’re away! :-)

Finally, in the unfortunate event you need to restore from that location, you can use the GUI to do this.

  • Select complete PC Restore
  • Select restore a different backup
  • Select advanced
  • Specify the ocation of your backup

But of course we hope you never need to know that! ;-)

Mar 23

Sometimes, you have a MySQL database which contains duplicate records and you wish to find them. This code will be useful to you:

SELECT COUNT( table1.column_name) , table1.column_name
FROM table1
WHERE {whatever your where clause is, if required}
GROUP BY table1.column_name
HAVING COUNT( table1.column_name ) >1

And that’s it. You will now see your duplicate records.

Nov 11

When you use Group Policy Objects to redirect users folders, such as Documents, Downloads, Contacts, etc (as you should do), there is a drawback – when a user logs in for the first time and it makes their folders, the domain\administrator has no access to those files.

This can make getting to the users data for a variety of reasons (i.e. backups!) very difficult and painful and there is no GUI way built into Windows that is much help here. icacls is not, by itself, useful, as you still need to gain ownership of a folder to give yourself access to files and inherited permissions will be a problem as the GPO will give the root folder full access to “domain users”, meaning icalcs will inherit full access to effectively everyone, if you’re not very, very careful.

So I have made this vbscript to give just the user in question, the administrator and the system full access to each user folder, all at once. This script can be re-executed before any backups, to ensure you have access to all the files.

Here we assume all user profiles are stored in “e:\users”. The script is below:

set WshShell = WScript.CreateObject("WScript.Shell")
strComputer = "." Set objWMIService =
Set objWMIService = GetObject("winmgmts:{impersonationLevel=impersonate}!\\" & strComputer & "\root\cimv2")

Set colSubfolders = objWMIService.ExecQuery("Associators of
{Win32_Directory.Name='e:\users'} Where AssocClass = Win32_Subdirectory
ResultRole = PartComponent")

For Each objFolder in colSubfolders

    Wscript.Echo "Fixing " & objFolder.Name & " ...... "

    userToFix = replace(objFolder.Name,"e:\users\","")

    wscript.echo "User: " & userToFix wscript.echo "TAKEOWN /f E:\users\"
& userToFix & " /r /d y"

    Return = WshShell.Run("TAKEOWN /f E:\users\" & userToFix & " /r /d
y", 1, true)

    wscript.echo "icacls e:\users\" & userToFix & " /grant domain_name\"
& userToFix & ":(NP)(F) /T"

    Return = WshShell.Run("icacls e:\users\" & userToFix & " /grant
domain_name\" & userToFix & ":(NP)(F) /T", 1, true)

    wscript.echo "icacls e:\users\" & userToFix & " /grant
domain_name\administrator:(NP)(F) /T"

    Return = WshShell.Run("icacls e:\users\" & userToFix & " /grant
domain_name\administrator:(NP)(F) /T", 1, true)

    wscript.echo "icacls e:\users\" & userToFix & " /grant SYSTEM:(NP)(F)
/T"

    Return = WshShell.Run("icacls e:\users\" & userToFix & " /grant
SYSTEM:(NP)(F) /T", 1, true)

    wscript.echo "User fixed"

Next

Now all users will still have full access to their files and you will be able to get full access to all their files as well. No other users will have been given a way in, so it remains secure.

Oct 20

Recently I was trying to join a Windows 2008 R2 64 bit server to an existing Windows 2003 based domain, as a new domain controller. The old domain controller was Windows 2003 R2. When I attempted to run dcpromo on the new 2008 R2 server, I was told that I needed to run “adprep /forestprep” first.

This file, adprep.exe, is located on the Windows 2008 R2 CD, under the support folder but must be executed on the existing schema master of the domain (i.e. in my case, the Windows 2003 R2 32bit domain controller).

Upon running the file, I found that I was given the message:
"adprep.exe is valid, but is for a machine type other than the current machine."

This rather cryptic message basically means that the adprep is for a 64bit OS and the Windows 2003 Domain Controller is 32bit. I of course Goolged this error and was given a hotfix from Microsoft to resolve it. The hot fix is actually just a new version of adprep. Unfortunately, it gave the same message.

In the end it turns out there is a file called adprep32.exe sitting in the same folder as adprep.exe on the CD – I didn’t see it because I was using a command prompt to run the tasks above and never looked around the folder. Silly mistake. Running this file upgraded the Schema from 33 to 47, which is required for Windows 2008 R2 and allowed me to promote the new domain controller.

P.S. You can check the existing schema with the command:
Schupgr.exe

May 27

I’ve recently purchased a new server for our production environment and went to our preferred provider of hardware for a new server. The one stipulation I had was that is was VMWare ESX certified. I did my research and selected all my components and then purchased the server. However, after purchasing, the vendor sent me an email stating they were substituting some of the components on me. Of course, I panicked about the VMWare issue, as drivers for VMWare are not like Windows – you can’t get them from hardware sites, more often than not. Anyway, the vendor assured me the subs would still be ok.

Long story short, although they kept to the letter of the law on that, they didn’t keep to the spirit of the law, so to speak. All the components are VMWare certified – except the very expensive hardware RAID card cannot be used on the VMWare ESX OS partition. In otherwords, VMWare can still have the guests in a mirror, but the OS will be outside the RAID.

This left me with 3 choices:

  • 1. Have the VMWare OS outside the RAID and hope the disk doesn’t fail. Not a great idea on a production server.
  • 2. Install Linux on the server and set up a RAID, then run VMWare Server 2 and put guests in that. Works ok but the disk I/O hit is significant and it’s not a bare metal solution.
  • 3. Switch to Hyper-V.

I’ve always been meaning to try out Hyper-V but a production-acid-test is not what I had in mind. None the less, I am an impulsive idiot, so I am going to do it. While it’s production, it won’t directly effect any of our customers and I can wear the headache myself, for the time being, if it’s not great.

So anyway, back to the topic of the post.

I began the installation of Microsoft Hyper-V server on the box and got as far as being asked for the RAID card drivers. After providing these, I was dismayed to see that Windows was telling me it was able to see the disk, create the partitions and format them but was still “unable to install to the selected location”.

This didn’t seem fair. I have a driver and OS that can see the disks, I can work with the disks but I can’t use them? This sucks!

Being the old school guy I am, I of course went looking for newer drivers but it turns out I already had the newest. I was almost ready to give up and go for option 2 above, when I came across a little trick.

It turns out that this error message is kind of Windows’ weird way of saying “please put the Windows CD back in the drive and remove the driver CD”. It actually turns out all you need to do to continue is pull out the driver disk and put the Windows 2008 R2 DVD back in the drive, then rescan the disks. Hey Presto! You can now install to the partitions.

Personally, I find this a little silly but at least I can move on and try out my Hyper-V set up now.

I’ll let you know if I live to regret it.

PS: I understand this issue only effects 64bit installations of Windows 2008 / Windows 7. I may be wrong on this but have never tried a 32bit install and have no plans to do so.

Feb 9

It’s probably not a bad thing but if you’re a Windows 2000 Server (or Professional for that matter) user, it’s time to start thinking about upgrading because Windows 2000 is about to “end of life”.

Microsoft have put online a Windows 2000 End of Support Solution guide (which pretty much amounts to “please buy some more software”) here.

Dec 21

See this news over at Rustylime: Firefox 3 is now the #1 browser in the World.

Browser usage stats

Browser usage stats

Why not add your voice to their vote:

Dec 2

Giving elevated permissions to non administrative users can be a real pain, in Windows land. In fact, it can be so much of a pain that people often don’t bother, and just elevate users to have admin rights when they don’t need them.

A good example of this is when someone needs to be able to restart a service. Often, vendors need access to work on one of their applications on a server – and as part of that, they need to be able to restart a service. However, you don’t really want to hand them the keys to the server at the same time.

So instead of simply giving them local admin rights, or spending ages working with NTMon and NTRegmon to try to work out the specific reg keys and file permissions to change, how about just using Microsoft’s SubInACL tool?

It’s actually very easy to use. Basically, it allows you to modify the security settings directly related to just one service, giving an non-privileged user the ability to work with just that one application (although you can actually do much more with SubInACL than just services).

Here’s the syntax to give a user called “user1″ access to “service1″ on the server “\\domain1\server1″.

subinacl /service service1 /grant=domain1\user1 = TO
Here, T = start service and O = stop service.

Other commands include:
F : Full Control
R : Generic Read
W : Generic Write
X : Generic eXecute
L : Read controL
Q : Query Service Configuration
S : Query Service Status
E : Enumerate Dependent Services
C : Service Change Configuration
T : Start Service
O : Stop Service
P : Pause/Continue Service
I : Interrogate Service
U : Service User-Defined Control Commands

SubInACL can be downloaded here.
Please note earlier versions from the original Windows 2003 Resource Kit will not work on patched / service packed Windows 2003 servers, so this is the new version.

Nov 17

I recently ran a yum update on my CentOS 5.2 server, which I use to run VMware guests – really you could call it my VMware server. Every time I do this, I think to myself “should I really be doing this? Do I really need the newest kernel on this box that works perfectly?” Then I do it anyway and hope for the best.

This time, however, things did not go for the best.

Immediately after updating and rebooting, VMs went to hell. Guests would suddenly stop for no reason, the web interface kept failing and not responding (although the vmware-hostd service would continue to listen on the ports). I assumed it must be kernel related and dropped back to the last one – but to no avail.

Having no time to fix the issue, and all my VMs on a separate set of disks, I simply reinstalled CentOS from scratch, using the very latest version – 5.4. I thought to myself, “this will surely fix the problems – it’s a fresh OS”. Wrong

Long story short – glibc-2.5-42, which incidentally ships with CentOS 5.4 out of the box, is incompatible with VMware.

Here’s how to get VMWare working on your CentOS 5.4 host:
1. Go to /etc/yum.repos.d and copy the file CentOS-Base.repo to a new file. I chose CentOS53-Base.repo.

2. Edit this new file and rename all instances (in [ ] brackets) with a new name. I.e.
[base] becomes [base53]
[updates] becomes [updates53]
etc

3. Replace every instance of $releasever with 5.3.

4. Save the file. Now downgrade glibc:
yum downgrade glibc glibc-common

5. Edit the /etc/yum.conf file and exclude these libraries from future updates, until this is resolved. Do this by adding the following line:
exclude=glibc glibc-common glibc-devel glibc-headers nscd

6. Reboot and re-run vmware-config.pl.

Everything should now work, once more.

Nov 15

There’s lots of information around, relating to how to improve VMware performance; some of it useful and some of it not so useful.

For example, it does help to be reminded to defrag your guest OS virtual hard disks – but being told that fibre channel is faster than DAS is a little pointless, when you’re trying to improve the speed of your personal VMware box and don’t have the budget for such things.

So with this in mind, I’ve collected bits and pieces that I have found made a big difference to performance, without having to spend money on hardware.

Firstly, the Guest OS:
1. As mentioned above, where appropriate, always keep the virtual disks de-fragmented.
It does make a small improvement to performance and disk size. Obviously, the more fragmented the disk, the worse the performance and the bigger the file size.

2. Use the minimum number of CPUs per guest OS.
Contrary to “real” computers, more processors does not equal more power. In fact, it can equal less power. If you have a dual or quad core host system, synchronising your cores across the guest OS, with the host OS’s use of them is actually likely to degrade performance, not enhance it. A CPU can only process one thing at a time (albeit pretty damn quickly), so each instruction has to queue and be run in synchronisation. It’s likely that if you try your guest with just one CPU, it will perform better than if you try it with two. Especially if you are using more virtual CPUs than you have physical.

3. Always choose the SCSI disks over the IDE.
As can be seen here, a virtual IDE controller will use more CPU load than a virtual SCSI controller.

4. Always install VMware Tools on the guest OS.
It allows the guest to stay in better communication with the host and will help.

5. Pre-allocate disk space.
It’s faster but it makes the storage requirements more of a problem. Your call.

6. Disable anything you don’t need.
A guest VM does not need visual pretties, high res wallpapers, screensavers, etc. Turn them off. You don’t need a fancy Matrix-style screensaver that requires 40% of the CPU and 512MB of RAM, on a screen no one can even see. Any services you don’t need, as well.

7. No RAID.
There is no sane reason why the guest OS needs virtual disks in a virtual RAID. This makes no sense – they’re almost certainly just files stored on the same physical disk or RAID anyway – let the Host OS worry about RAID.

8. Don’t use Snapshots.
They really slow Guest OS systems down.

Speeding up the Host
There’s only so much you can do on the guest, to make an impact, if the host is running slow. So let’s address what we can, there, as well. You’re likely to find a bigger impact on the Host than the guest.

1. Set up the hard drives in the most efficient, fastest fashion.
Obviously, a slow software RAID is going to degrade performance – but you may not have a choice. I myself use a software RAID-1 on my VMware Host because I am willing to sacrifice some performance for peace of mind and i don’t have the money for a real RAID card – but I acknowledge the performance will be degraded.

2. Don’t over use resources.
You have a finite amount of RAM and number of CPUs. Deal with it. Don’t allocate 10 VM guests, with a CPU and 2GB of RAM each on a dual core PC with 4GB of RAM and then complain things are slow. If you care about performance, then allocate 1 CPU to the Host OS, and 1 CPU per guest. Ensure the Host OS has enough RAM to do its job and don’t allocate more RAM than you have, physically.

3. The Host OS must be minimised.
The Host OS should not need to run a web server, etc, etc. It should be a VMware server. That’s all it should be doing. Don’t install junk you don’t need and, if possible, don’t use Windows as the Host OS. This is because you can squeeze a lot more performance out a Linux host, with a much smaller footprint. You won’t need to run anti-virus products, etc, which all hog memory and CPU cycles.

4. The Host OS should be maximised.
Maximise the performance of the Host OS and the way VMware Server 2.x interacts with it. See below for practical ways to do this, right now, with next to no effort. They make a very noticeable performance.

5. Disable power management on your Host OS.
If you’re running Linux, compile the kernel to use noapic, (you may need to do this anyway, on an AMD chip). See the link (also on this blog) as to how to do this.

6. Synchronise the CPU speed on the Guest and Host systems.
See this entry (on this blog) as to how to do this.

7. Set the Host system BIOS up for maximum performance.
There will almost certainly be an option in your BIOS for maximum performance. Basically, this sets the CPU timer to be constant, not cycling down to conserve heat / power, etc.

8. Separate physical disks for OS and VMware Guests.
This way, the OS read / writes don’t interfere with the VMware guest disk I/O. Obviously, the faster the disks, the better.

9. Make some config changes to your Host VMware Server.
Out of the box, VMware works fine – but performance isn’t great – especially disk I/O. The below changes make significant improvements over default performance.

Changes to the VMware configuration (assuming Linux Host OS).
1. Edit the /etc/vmware/config file. Add the following:
tmpDirectory = "/tmp/vmware"
mainMem.useNamedFile = "FALSE"
sched.mem.pshare.enable = "FALSE"... Read more
MemTrimRate = "0"
MemAllowAutoScaleDown = "FALSE"
prefvmx.useRecommendedLockedMemSize = "TRUE"
prefvmx.minVmMemPct = "100"

Basically, what we’ve done here is set a new temporary directory for VMware to use (we’ll come to this in a minute). Then we’ve told it use no more memory than we actually have and to stop sharing it between guests (much faster), not allow memory trimming and cut out sharing of certain resources.

2. Give VMware it’s own temporary location (swap) for files.
As you see above, we’ve defined a new location for VMware to keep it’s swap info, instead of in the regular file system. Aside from the above, we now need to make a location “/tmp/vmware“. We then create the following entry in /etc/fstab, so we can define this location as new file system:
tmpfs /tmp/vmware tmpfs defaults,size=100% 0 0
Now Linux knows this location is a temporary file system – think of it a bit like a swap file.

Once you make the changes above, you can expect to see a reasonable performance improvement, as opposed to a “vanilla” VMware Server 2.x environment. I can personally attest to a really noticeable disk I/O improvement, especially on Windows based Guest systems.

« Previous Entries