Search This Blog

Thursday 23 December 2010

Blackberry Express Server Web Console will not load after Maintenance Update 5.02

If you are intending to deploy the BES Express Server Maintenance Update 5.02 to your BES Server 5.01 then make sure you do so as the BESADMIN user. If you install the update as any other user the update will install correctly but you will find that the Blackberry Administration Service Console will not load and Internet Explorer will display "Page cannot be displayed".

I suffered this myself preferring to make installations with a specific Domain Administrator account and found that my console was no longer functioning. After much searching on Blackberry Support and finding others in the same situation via Google a colleague suggested he had found that Blackberry updates should always be installed as the BESADMIN user.

It is not really how the idea of security groups should work but it will save you no end of head scratching if you make sure you use the BESADMIN account, if you have already installed as another user just re apply as BESADMIN and the issue will be resolved.

Wednesday 15 December 2010

High Host CPU on Windows 2000 Virtual Machine in vSphere

I had a Windows 2000 Server that I had used a Physical to Virtual migration with the VM Converter tool, the original physical server had two CPUs and when I mgrated I dropped this to one vCPU as the server needed very little resource.

The problem arose that the Windows 2000 Server was consuming a high percentage of the host CPU usage while the VM Guest itself as idling at 0% CPU usage. The problem I found by researching the VMWare KB articles is that the Windows 2000 Server will try to set itself into idle mode when not in use, in a physical server this results in low power consumption, in a virtual server this results in low CPU usage on the host.

However my server was not able to enter idle mode and so was stuck trying to do so and was consuming a high percentage of the host CPU. The issue was that I had not changed the ACPI HAL for the server in Windows 2000 to tell it it was no longer a dual CPU server, I changed this by entering the Device Manager and then expanding the Computer object, below this is the ACPI Multi Processor object.

If you enter the properties of this object you can from the Advanced Tab choose to update the driver, here you choose to select the driver from a list and show all hardware that is compatible. The list will then show the Uni Processor HAL object that can be used.

I added this object and then restarted the server and the CPU use on the host has now reduced to a normal idle level.

Friday 10 December 2010

Reset Alarms in vSphere 4.1

I had an alarm trigger recently on a Windows 2000 VM I have in a vSphere 4.1 Cluster, the CPU spike was only temporary and I could see that the actual cpu usage had returned to normal but the alarm was still showing in Vcenter and I could not clear the alarm.

This can be done simply by finding the parent for the alarm and then disabling and re enabling the alarm, it looks like a possible bug in vSphere so this is a quick work around that does not have any effect on the live service.

Thanks to http://www.virtualizetips.com/2010/04/how-to-clear-alarms-in-vsphere-vcenter-4/ for the information.

Tuesday 7 December 2010

HP MSA 2324i SAN Array Controller Unavailable

We had deployed a new HP MSA 2324i SAN Array recently and after about 2 months we found Controller A to be unavailable, the storage network was still functioning but we could not access the Controller A.

We connected to Controller B and could see that Controller A was offline but we could not restart or shutdown the controller, the event log showed that the Controller had been taken offline and could not be restarted.

I had the Controller replaced under our support with HP and when we installed the replacement we checked the firmware and could see that the latest firmware had a critical status that explained that the controller would actually become degraded by the firmware unless we upgraded.

This is explained in this HP article.

If you have any MSA 2000 array series SAN then you must upgrade the firmware or risk the loss of both Controllers in the future.

Tuesday 30 November 2010

Send As permission for Exchange 2007 Distribution Group

It can be quite handy to be able to send an email as a distribution group sometimes, allowing several users in a group to appear as one generic department email.

The task is a little different to Exchange 2003 as there is no GUI available for this task and it must be completed with the EMS.

The command you need is

Get-DistributionGroup groupname | add-adpermission -user username -extendedrights -Send-As

Once done you then have to wait, it can take some time for this change to propagate so be patient!

Monday 29 November 2010

VMware vCenter Error Call “PropertyCollector.RetrieveContents” for object “propertyCollector” on vCenter Server failed

I had this error randomly when working on vSphere Client 4.1 to a vSphere 4.1 installation. It turns out this is caused because I had an ISO image mapped to a VM CD drive and the ISO file was no longer in the datastore. I removed the ISA from the VM settings and the problem went away.

Thanks to Yuri's Technology Blog for this, I just wanted to re post this.

http://yuridejager.wordpress.com/2010/07/24/vmware-vcenter-error-call-propertycollector-retrievecontents-for-object-propertycollector-on-vcenter-server-failed/

Monday 22 November 2010

Group Policy Management in Windows 7

Don't forget that if you need to make changes for Windows 7 in your Group Policies that you need to download the Remote Administration Kit for Windows 7.

http://www.microsoft.com/downloads/en/details.aspx?FamilyID=7d2f6ad7-656b-4313-a005-4e344e43997d&displaylang=en

Friday 19 November 2010

Measuring SQL Server Performance

I had a customer with a reported slow SQL Server, they run a MRP application and I have several customers with the same SQL database so I had a good bench mark to check against.

I used two tools for this Process Monitor and the now defunct Filemon, both from SysInternals.

I used Process Monitor to look at the disk I/O reads and writes and noted that the customer with the performance issues had disk I/O reads of 88 million and writes of 44 million. I then used Filemon to view the disk read/writes on the SQL Server database disk and I could see up to 400 disk transactions per second.

I compared this to the customer with no problems and they returned 16 million reads and 16 million writes with 15 disk transactions per second using Filemon.

Although there is more to this type of investigation such as the RAID array type, disk spindle speeds, nature of the MRP use and age of the physical servers, I found that this was a good method of getting a good overview of the disk usage and if the disk I/O is the bottleneck on your SQL Server when this type of issue arises with performance.

Security warning when you start Outlook 2007 and then connect to a mailbox that is hosted on a server that is running Exchange Server 2007 or Exchange Server 2010: "The name of the security certificate is invalid or does not match the name of the site

If you run SBS Server 2008 and you have a third party SSL Certificate for your Exchange Services you can find this error will occur when users running Outlook 2007 or Outlook 2010 connect to the Exchange Server. Microsoft provide a KB for this issue here

http://support.microsoft.com/kb/940726

However the KB references a standard Exchange Server installation and not SBS Server and so the reference to IIS is incorrect.

Ensure that you change the reference to the Default Web Site to the the IIS site that contains the Exchange IIS information, SBS Web Applications.

My server for example is as follows

Set-WebServicesVirtualDirectory -Identity "SBIZ\EWS (SBS Web Applications)" -InternalUrl https://remote.mydomain.co.uk/ews/exchange.asmx

Set-OABVirtualDirectory -Identity "SBIZ\oab (SBS Web Applications)" -InternalUrl https://remote.mydomain.co.uk/oab

Set-UMVirtualDirectory -Identity "SBIZ\unifiedmessaging (SBS Web Applications)" -InternalUrl https://remote.mydomain.co.uk/unifiedmessaging/service.asmx

Saturday 13 November 2010

Manually uninstall Trend Micro Office Scan Client

Sometimes you cannot uninstall the Office Scan Client so here is the way to uninstall it manually.

1.Go to Control Panel Services (services.msc), and stop the following services:
◦OfficeScanNT Listener
◦OfficeScanNT RealTimeScan
◦OfficeScanNT Personal Firewall (if enabled)

2.Run Registry Editor (regedit.exe)

3.Navigate to the following registry key hive:
HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Services

4.Delete the following keys (if available):
◦Ntrtscan
◦Tmlisten
◦TmFilter
◦VSApiNt
◦TMPreFilter
◦TM_CFW
◦OfcPfwSvc

5.Navigate to the following registry hive:
HKEY_LOCAL_MACHINE \SOFTWARE\TrendMicro

or

HKEY_LOCAL_MACHINE \SOFTWARE \Wow6432Node\TrendMicro (in 64-bit Windows operating system)

6.Delete the following keys (if available):
◦OfcWatchDog
◦Pc-cillinNTCorp or OfficeScanCorp (depending on the client)
◦RemoteAgent
◦PC-cillin
◦CFW

7.Browse to the following registry key hive:
HKEY_LOCAL_MACHINE \SOFTWARE \Microsoft\Windows\CurrentVersion\Run

8.Delete the OfficeScanNT Monitor key.

9.Navigate to the following registry key:
HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\Windows\CurrentVersion\Uninstall

10.Delete the OfficeScanNT key.

11.Delete the OfficeScan program group (Trend Micro OfficeScan Client) from the Windows Start menu.

12.Restart the computer.

13.Delete the directories that contain the OfficeScan Client program files, normally located inside Program Files folder.

Sunday 7 November 2010

WIndows 2008 Event ID 39 & 17 Terminal Services

I had a Windows 2008 Terminal Server that had had the certificate expire and I was getting multiple errors issuing Terminal Server licences to the clients. It turns out to be a known Microsoft issue

http://support.microsoft.com/kb/2021885

Friday 5 November 2010

Veritas Backup Error "0xe00084af - The directory or file was not found or cannot be accessed"

I have a customer who still runs Veritas Backup Exec version 9.1 and I had recently upgraded WSUS to version 3, I noticed that the overnight backup was now failing each time with the error in the post title.

If I tried to backup any file from the C: drive this error would occur event though the files were accessible in Veritas, a backup of the Information Store or Shadow Copy was successful.

The cause of the error is the SQL Server VSS Writer Service that is installed as part of WSUS 3 application and if this service is running it will cause the Veritas Backup to fail.

All you need to do is use the Pre and Post Backup files to stop and start the service and you can continue to backup as normal and avoid this rather ambiguous error.

Thursday 28 October 2010

Sage SBD SoftwareInstallation.UI.EXE 100% CPU

We had a Terminal server running Sage Line 50 2009 that was showing 100% CPU on a regular interval, task manager showed that the process Sage.SoftwareInstallation.UI.exe was taking the CPU time.

This process is part of the auto update to Sage Line 50 and should not be ran by a standard user, but this Sage application will do so as it is no strictly Terminal Server aware.

The way to stop this process running is to rename the EXE files and the associated DLLS that are stored in
C:\Program Files\Common Files\SageSBD

Sage.SDB.Platform.Installation.SoftwareUpdates.dll
Sage.SDB.Platform.Installation.SoftwareUpdates.Common.dll
Sage.SDB.Platform.Installation.SoftwareUpdates.Model.dll

Thanks for http://www.bleedyellow.com/blogs/YellowNotes/?lang=en for the information!

Tuesday 26 October 2010

SBS 2008 Outlook Anywhere and Terminal Services Gateway

I have had a problem with a SBS 2008 Server where I needed to provide Outlook Anywhere and Terminal Services Gateway for users with one SSL Certificate FQDN.

I had the SSL Certificate enabled in Exchange 2007 and testing this I could access a mailbox from an Outlook client at a remote location. This was using a SSL Certificate from www.digicert.com

The problem came when I enabled TS Gateway on the same server to publish an internal Windows 2008 Terminal Server. Everytime I tried to connect I would get asked for credentials and then once entered the process would repeat.

The problem stems from the fact that this is a SBS 2008 Server and TS gateway and Outlook Anywhere share the same IIS website and you have to make a few changes to enable both services.

In Exchange Management Console I changed Outlook Anywhere authentication to use NTLM from basic, this is because both TS Gateway and Outlook Anywhere cannot use the same authentication and by default Outlook Anywhere uses basic authentication and TS Gateway will use Windows Integrated Authentication.

If you mix the two you get this Event ID in the Application Log

Event 3003 MsExchange RPC over HTTP Autoconfig

The Outlook Anywhere authentication settings have been updated.

Old settings: Basic, Ntlm
New settings: Basic


This is because Exchange will change the authentication back to Basic only for the RPC virtual website in IIS when TS Gateway changes it to use Windows Integrated Authentication. If you wait 5 minutes then Exchange reverts the changes that the TS Gateway MMC makes and you cannot logon with TS Gateway.

The solution is to enabled NTLM authentication in Exchange Management Console for Outlook Anywhere and then in IIS under the RPC virtual site enabled Windows Authentication manually.

Now that you have NTLM for Outlook Anywhere, Exchange will not try to change the authentication for the RPC virtual site back to basic and the Windows Authentication setting remains and TS Gateway works as expected.

Sunday 17 October 2010

Cisco 1841 EEM command not executing

I have a configuration on Cisco 1841 routers that perform a failover for dynamic NAT if one DSL circuit fails, using a IP SLA Track I have a ping set to a IP such as 8.8.8.8 via one interface and if this ping drops then this calls an EEM applet to clear down the NAT translations and the secondary route for 0.0.0.0 takes over and a route map handles the PAT for the clients.

Well the problem was that the failover was working correctly but when the EEM applet fired it did not clear the NAT translations, if I ran the command manually from EXEC mode then I could see the NAT translations rebuild on the failover DSL interface and the Internet was available to the clients again.

Digging around on the EEM I found a debug command to debug the EEM events

debug event manager action cli

Now when I tried my failover by administratively shutting down interface atm0/0/0.1 I could see the events occur in the logs. Now I could see clearly that the cli command starts in USER mode and hence cannot complete a EXEC mode command! All it needed was a new line to add enable to pass the cli command into EXEC mode. The EEM applet is below to show this

event manager applet failover
event track 1 state any
action 1.0 cli command "enable"
action 1.1 cli command "clear ip nat translation *"
exit

By using enable and entering EXEC mode the command completes and the NAT translations are rebuilt in under three pings!

Friday 8 October 2010

Move Public Folders when SSL Certificate has expired

I was migrating a Small Business Server 2003 Server to Small Business Server 2008 and during the Public Folders migration I found that the source server SSL Certificate had long expired. When I tried to move the replicas to the new Exchange Server the error "the received certificate has expired" was shown.

I check the certificate and it was expired by over a year and so the next step was to remove the SSL certificate from the Exchange Server as this had had all the mailboxes moved and was no longer a CAS. I removed the certificate in IIS and then noted when I tried to move the replicas that the error "the handle specified is invalid" was shown.

This is displayed because the Exadmin virtual website in IIS was set to use SSL on the Directory Security tab, I unchecked the boxes for Use SSL and Use 128bit Encryption and I was then able to use the ESM to move the replicas and migrate my Public Folders.

Saturday 2 October 2010

SBS 2008 & Windows SharePoint Services 3 Search Event ID 2426 Error

I have found on a default SBS 2008 Server installation that an error will always appear in the Application Event Log for the Source Windows Sharepoint Services 3 Search.

Event ID 2424

The update cannot be started because the content sources cannot be accessed. Fix the errors and try the update again.

Context: Application 'Search', Catalog 'index file on the search server Search'

The cause for this is described by Microsoft as

"You receive above warning events because WSS3.0 Search service is trying to crawl the WSS content via the URL – remote.domain.com, which is mentioned in above event. Windows Server 2008 includes a loopback check security feature that is designed to help prevent reflection attacks on your computer. Therefore, Kerberos authentication on Default Content Access Account fails if this URL does not match the local computer name and is not registered in system as additional Service Principle Name (SPN)."

The resolve for this is to disable the Loopback Check in the Registry.


[HKLM\System\CurrentControlSet\Control\Lsa]

"DisableLoopbackCheck"=dword:00000001


Restart the Sharepoint Services Search Service and the error will stop occurring.

Friday 1 October 2010

Poor Terminal Server Performance and Event Log 11728

I have had several Terminal Servers that have displayed poor performance including CPU spikes, application faults and slow printing. All have shown the MSIEXEC event ID 11728 in the Event Log. This is related to a HP Printer Driver that causes a fault if the user is a non admin, which will be all your Terminal Server Users!

HP do have an official line on this below.

HP UPD/Discrete LaserJet Driver - Microsoft Windows Event Viewer (Application) Message "HP CIO Components Installer" - hpzbdi32.msi / hpzbdi64.msi / hpzbdi.dll
Issue
Several reports of HP Bidirectional Channel components (hpzbdi32.msi /hpzbdi.dll , or hpzbdi64.msi /hpzbdi.dll ) could be the potential cause of various performance-related issues within standalone print servers, Microsoft Windows Cluster Server, Citrix, and/or Terminal Server environments, resulting in delays during driver installation, intermittent print spooler crashes, performance issues, CPU spikes, and so forth. CIO MSI does not get installed if a user with non-admin rights attempts to install the driver. The driver will attempt to install the MSI multiple times (five times, with an interval of two minutes between each try) without checking the access permission. This multiple execution causes a delay with the installation, and will launch multiple instances of the MSI installer, which can contribute to the cause of poor performance on the server.
The Microsoft Windows Event log (Application) will usually indicate a reference to: 32 Bit HP CIO Components Installer or a 64 Bit HP CIO Components Installer Product Version: X.X.X. Product Language: XXXX .
This issue may manifest itself with the discrete PCL 6, PCL 5, and Postscript drivers, which are bundled with hpzbdiXX.msi /.dll components. As hpzbdi is considered a legacy bi-directional component, there will not be any further updates or developments to this component. The workarounds that follow have been reported to resolve the majority of issues concerning hpzbdi.msi /.dll reports.
Product/Driver Packages Affected:
The following products have hpzbdi components installed as part of their driver package, and would qualify for the following workarounds:
• HP LaserJet 4250/4350 Printer Series
• HP LaserJet 9040/9050 Printer Series
• HP LaserJet 5200 Printer Series
• HP LaserJet P3005 Printer Series
Solution
The latest versions of hpzbdi.dll , which are not publically available at www.hp.com, and can only be procured by contacting HP Support, are as follows:
• 32-Bit: hpzbdi.dll (1.1.2.51)
• 64-Bit: hpzbdi.dll (1.2.2.51)
When the driver components are installed, the hpzbdi components are installed to the following paths by default:
• 32-Bit: C:\WINDOWS\system32\spool\drivers\w32x86\3
• 64-Bit: C:\WINDOWS\system32\spool\drivers\x64\3
NOTE: The hpzbdi files may also reside in additional folders named hewlett_packardhp_xxxxxx within the w32x86/x64 directories. It is not necessary to make changes to the hpzbdi files located in these directories, per the workaround instructions that follow.
Recommended Solution:
It is recommended to perform a clean installation/upgrade to the HP Universal Print Driver version 4.7.2, as it resolves the issue reported in this document. It should be noted that the HP Universal Print Driver PCL 6, PCL 5, and Postscript version 4.7.2 are not available at www.hp.com. Although this is a fully Microsoft certified driver, which has been through the Microsoft Windows Hardware Quality Labs (WHQL) certification, it is only available from HP Support.
NOTE: Click here to review additional information related to UPD installation/migration at www.hp.com/go/UPD.
It is possible a UPD upgrade to version 4.7.2 is not a viable option, due to logistics beyond the immediate control of the customer. Should this be the case, alternative steps can be taken to address this issue.
Workaround #1:
Zero-byte the MSI Installer Package, which will eliminate the symptomatic behaviors of the issue noted in this document. Complete the following steps:
1. Open Notepad and save the file as hpzbdi32.msi or hpzbdi64.msi . This zero-byte file will not contain any data.
2. Navigate to the %WINDOWS%\system32\spool\drivers\w32x86 (32-bit operating system) or %WINDOWS%\system32\spool\drivers\x64\3 (64-bit operating system) directory, and locate the hpzbdi32.msi or hpzbdi64.msi file. Rename the file to hpzbdi32.old or hpzbdi64.old .
3. Copy and paste the newly created zero-byte file named hpzbdi32.msi or hpzbdi64.msi into the %WINDOWS%\system32\spool\drivers\w32x86 or x64\3 directory.
4. Verify functionality.
NOTE: Depending on configuration, it may be necessary to stop the spooler, perform the steps noted, and then restart the spooler.
Workaround #2:
Complete the following steps:
1. Working with HP Support, obtain the hpzbdi hotfix , which contains either the 32-bit: hpzbdi.dll version 1.1.2.51 and/or 64-bit: hpzbdi.dll version 1.2.2.51.
2. Navigate to the %windows%\system32\spool\drivers\w32x86\3 directory for 32-bit operating systems, and rename hpzbdi.dll to hpzbdi.old .
3. Copy and paste the hpzbdi.dll version 1.1.2.51 hotfix to this working directory.
NOTE: For 64-bit systems, navigate to %windows%\system32\spool\drivers\x64\3 . For step 3 copy and paste the hpzbdi.dll version 1.2.2.51 hotfix to this working directory.
4. Verify functionality.
NOTE: Depending on configuration, it may be necessary to stop the spooler, perform the steps noted, and then restart the spooler.
How to Obtain Support for this Issue:
Customers: If you require the hpzbdi hotfix , as outlined in this document, please contact 1-800-HPINVENT and report the issue to the HP Support agent. Request the hpzbdi hotfix and refer to this document.
HP Support: If an HP customer contacts you to report this issue, and to request the hpzbdi hotfix , please contact your Resource team for immediate consultative action.

Thursday 30 September 2010

Renew Exchange 2007 SSL Certificate

Once your Exchange 2007 SSL Certificate is due for renewal you will need to perform the renewal process, this is slightly complex and requires the right steps to be taken.

1. First you need to request the new certificate from Exchange to generate the CSR that is needed for your SSL Provider. I use www.digicert.com as they have a nice wizard to generate the Exchange 2007 Shell command for you. In my instance I used this Shell command to generate my certificate

New-ExchangeCertificate -GenerateRequest -Path c:\mydomain.com.csr -KeySize 2048 -SubjectName "c=GB, s=West Midlands, l=Birmingham, o=My Company Limited, cn=mydomain.com" -DomainName server, server.mydomain.local, autodiscover.mydomain.com -PrivateKeyExportable $True

This now creates the CSR file you can send to the SSL provider for processing.

2. Once you have received your new SSL certificate you need to replace the expired or close to expired SSL certificate on your Exchange Server. You next need to run the Exchange Shell command

Get-ExchangeCertificate | fl | out-file –filePath c:\certs.txt

This will output your existing certificates to a text file you can read, open the file and find the certificate with the "Not After" date that is the expiry date of your certificate. Make a note of the thumbprint information and copy this to the clipboard.

3. Next you will remove the existing certificate with the command below and the thumbprint information on the clipboard

Remove-ExchangeCertificate –thumbprint

Confirm you want to remove the certificate.

4. Now you can import your new Certificate from the CER file supplied by your SSL Provider

Import-ExchangeCertificate -path e:\certificates\owa.cer –FriendlyName “owa.mydomain.com”

This will output the Thumbprint for the next Certificate and you need to make a note of this long file information.

5. You now need to enable the certificate for use in Exchange 2007 with the command

Enable-ExchangeCertificate -Thumbprint B52842F7408772B7151FF74FDAE914EA7B59B53A -Services IIS

Replace the Thumbprint with the Thumbprint from your previous notes.

6. The Certificate is installed and enabled now, you can run the command

Get-ExchangeCertificate

This will show you the certificate and that it is enabled for Web Access with the "W" in the Services section.

Monday 27 September 2010

Disable IE8 Customisation Page with Regedit

Sometimes on a Terminal Server you dont want that Splash Screen for IE8 showing as the end users cannot find which options to choose and start calling the helpdesk, so you choose to disable it for them.

This is simple with Group Policy if you have a Windows 2008 or Windows 7 PC but what if you are all Windows XP and 2003? Then you need a custom ADM or a registry edit to achieve this.

HKEY_LOCAL_MACHINE \ Software \ Policies \ Microsoft \ Internet Explorer \ Main

Reg Dword - DisableFirstRunCustomize = 1

The branches above will not exist in the Registry so you need to create the last two manually first.

Thursday 23 September 2010

SBS 2008 Migration fails with "Source Server does not meet requirements"

We were performing a migration from SBS 2003 to SBS 2008 and when the answer file was being checked during the upgrade we saw this error displayed and the list showed that the source server did not meet the requirements for Active Directory Schema and Exchange Server Service Pack level.

We knew that the source server was correct and the BPA had passed all tests so I thought it must be a communication issue from the SBS 2008 Server to the source server, I opened Wireshark and could see traffic from the SBS 2008 Server to the source server so the next step was to see if the SBS 2008 Server could ping the source server by Netbios and FQDN as these are part of the answer file.

During SBS 2008 Server setup you can press SHIFT + F10 and have a command line load, this is invaluable for trouble shooting and a quick ping showed we could not ping the Netbios or FQDN of the source server. I edited the HOSTS file on the SBS Server and updated it to resolve the source server correctly and ran the answer file check again....

Bingo! The tests passed and the installation continued. Lesson learnt, always test your DNS and WINS before you start. In this case my engineer explained he had had to change the IP range on the source server from a class B address to a class C address and this then told me DNS had not updated correctly and this is why DHCP on the SBS Server 2008 did not resolve the names correctly.

Monday 20 September 2010

Delete files older than a certain age

I had a need to delete some backup images files from a share after a period of time so that the disk did not run out of space. This was because the backup application Acronis & Backup Recovery 10.0 was not able to correctly clean up older image files in the backup rotation.

So to create a simple way to delete all image files older than two days I used the FORFILES.EXE utility from the Windows 2003 Resource Kit.

The syntax for this is quite simple and allows you to search a location or sub folder for a file type and then take an action depending on the files found. So for my case I used

forfiles /P E:\myPath /M *.TIB /D -2 /C "cmd /c del /q @path"

This command uses the path E:\myPath and then finds TIB file extensions, it tests if they are older than 2 days and if it matches it then runs a del /q to delete these files using the full path.

Wednesday 15 September 2010

NtFrs Event ID 13566

I had a server that had the event ID below

The File Replication Service has detected that the replica set "DOMAIN SYSTEM VOLUME (SYSVOL SHARE)" is in JRNL_WRAP_ERROR.

Replica set name is : "DOMAIN SYSTEM VOLUME (SYSVOL SHARE)"
Replica root path is : "c:\winnt\sysvol\domain"
Replica root volume is : "\\.\C:"
A Replica set hits JRNL_WRAP_ERROR when the record that it is trying to read from the NTFS USN journal is not found. This can occur because of one of the following reasons.

[1] Volume "\\.\C:" has been formatted.
[2] The NTFS USN journal on volume "\\.\C:" has been deleted.
[3] The NTFS USN journal on volume "\\.\C:" has been truncated. Chkdsk can truncate the journal if it finds corrupt entries at the end of the journal.
[4] File Replication Service was not running on this computer for a long time.
[5] File Replication Service could not keep up with the rate of Disk IO activity on "\\.\C:".
Setting the "Enable Journal Wrap Automatic Restore" registry parameter to 1 will cause the following recovery steps to be taken to automatically recover from this error state.
[1] At the first poll, which will occur in 5 minutes, this computer will be deleted from the replica set. If you do not want to wait 5 minutes, then run "net stop ntfrs" followed by "net start ntfrs" to restart the File Replication Service.
[2] At the poll following the deletion this computer will be re-added to the replica set. The re-addition will trigger a full tree sync for the replica set.

WARNING: During the recovery process data in the replica tree may be unavailable. You should reset the registry parameter described above to 0 to prevent automatic recovery from making the data unexpectedly unavailable if this error condition occurs again.

To change this registry parameter, run regedit.

Click on Start, Run and type regedit.

Expand HKEY_LOCAL_MACHINE.
Click down the key path:
"System\CurrentControlSet\Services\NtFrs\Parameters"
Double click on the value name
"Enable Journal Wrap Automatic Restore"
and update the value.

If the value name is not present you may add it with the New->DWORD Value function under the Edit Menu item. Type the value name exactly as shown above
.

I have had this error in the past and I have followed the registry edit to request FRS rescans for Active Directory changes, in this instance when I made the change the server had been in this state for so long it removed all files from the SYSVOL Share under SCRIPTS and POLICIES and rendered the server no longer a Domain Controller.

This was a pretty difficult situation as this was a SBS 2003 Server and this meant I had no other DCs to connect to bring AD back online.

I had to use the BurFlags registry changes to rebuild FRS again and start with an empty, to do this I had to perform an Authoritative Restore from this Microsoft KB article.

http://support.microsoft.com/kb/290762

Once this was complete I had the SYSVOL share back again after about five minutes but I had to manually recreate the SCRIPTS folder for this to be shared as NETLOGON. The Policies folder was missing from SYSVOL so I had no Group Policies and numerous UserEnv errors in the event log.

The next step was to rebuild the Default Domain Controller and Default Domain Policy, this can be done using the DCGPOFIX tool on the SBS Server, I ran this and accepted the Disaster Recovery options and this rebuilt the two Group Policies correctly.

Now I had to recreate all the SBS Custom Policies that no longer existed, for this I was able to export the policies from an existing SBS 2003 Server and then use the Import Settings option to rebuild the policies. As the polices do not reference any Security SIDS directly these policies could be imported.

This now returned the server to a state where the Event Logs were clear of errors, Exchange and SQL Server were working and I had only to recreate the logon script files in the NETLOGON share.

Thursday 2 September 2010

Change IP address from the command line

Here is a very simple way to change the IP address from the command line, very handy when you cannot get into the GUI and only cmd will load from task manager.

http://support.microsoft.com/?kbid=257748

Saturday 7 August 2010

Archive files with Robocopy

I have a few customers who keep all their data from the first server they ever had so we are looking at data from the 1990s onward! It always strikes me that if you don't need to keep data for legal reasons then if it hasn't been accessed for 6 months its ready to be archived off the server to NAS and then to offsite storage.

Robocopy is a tool that can help with this process, it can perform all sorts of file copy operations and in this case can move files that have not been accessed since a date to a new location and keep the file structure.

robocopy c:\share d:\archive\share /S /SEC /MOV /MINLAD:20081231 /L

This command will use the switches

/S - copy all subdirectories that contain files
/MOV - move the files and folders from the source
/MINLAD:YYYYMMDD - find files that have not been accessed before this date
/L - test the operation before you copy for real, very handy!

Monday 2 August 2010

Non admin users cannot logon to Citrix PS4.5 with RDP

I had an issue where we had to logon some users to a Citrix Presentation Server 4.5 via RDP as the VPN tunnel from their remote site was down, it was a temporary workaround but when we connected via RDP we had this error displayed.

"Connection Error : The desktop you are trying to open is currently available only to administrators"

This is related to the Terminal Server Configuration and the RDP Listener, under the Citrix Settings is a check box that says "Non published applications for Administrators only". This means the Desktop as this is a non published application and in my case my users needed the Full Desktop.

I removed this check box and they could logon as normal with RDP.

Sunday 1 August 2010

SSH to Cisco 1841 using route-map statements for PAT

I had a problem when using the Cisco 1841 router, I could not get access to the router from a remote location using ssh to the routers WAN IP.

I had been able to do this on the Cisco 877 but could not make it work on the 1841 router, after some head scratching and forum posting I was given a clue when I looked at the firewall logs. The logs showed that the packets were going to the router ok but on the return they were coming back from the wrong ports and this caused the packets to be dropped.

The firewall log is displayed here

%FW-6-DROP_PKT: Dropping tcp session 78.xx.xx.xx:3 86.xx.xxxx:45369 on zone-pair ccp-zp-self-out class ccp-icmp-access due to  Invalid Flags with ip ident 0

The log shows that the return packet has a source port of 3 but I know the connection entered on port 22 for ssh, so this means something has changed the packet source port before its return.

The answer is that NAT/PAT is involved and this is changing the packets on the return by PAT the packets back out of the ATM interface. The 8141 routers all use a route map so I can have two PAT statements and use the failover for the two ATM interfaces. So when the connection is made with ssh on port 22 the PAT statements are translating the traffic back out and this violates the ZBF rules and the packets get dropped.

To resolve this you have to use an ACL to allow the traffic you want PAT and deny everything else.

The current PAT statement and route maps are

ip nat inside source route-map O2 interface ATM0/0/0.1 overload

Route-map O2 permit 10
Match interface ATM0/0/0.1

So this route map needs to have an ACL added to only allow the traffic from my internal networks to be PAT. So I created a new ACL rule below

Access-list 120 ip permit 192.168.110.0 0.0.0.255 any
Access-list 120 ip deny any any

Route-map O2 permit 10
Match interface ATM0/0/0.1
Match ip address 120

Now that this ACL is added to the route map when the connection is made on port 22 the return traffic is matched against the route map ACL and this can see that the source IP is not in the 192.168.110./24 subnet so it is denied from being PAT and returns out of the ZBF with the correct source port and meets the ZBF inspection.

Friday 30 July 2010

GFI Mail Essentials Debug Logs and disk fragmentation

I often find that the disk which has GFI Mail Essentials installed has fragmentation issues and the files that are fragmented are the Debug log files. These files are not needed unless you need to debug, but the setting is on by default and causes your disk to become fragmented.

I switch these off unless I need to debug and keep my drive from fragmenting. You can switch the debug logs off in the registry here

GFI MailEssentials version 2010(x86): - [HKEY_LOCAL_MACHINE\SOFTWARE\GFI\ME15\Config]
GFI MailEssentials version 2010(x64): - [HKEY_LOCAL_MACHINE\SOFTWARE\WOW6432NODE\GFI\ME15\Config]
GFI MailEssentials version 14(x86): - [HKEY_LOCAL_MACHINE\SOFTWARE\GFI\ME14\Config]
GFI MailEssentials version 14(x64): - [HKEY_LOCAL_MACHINE\SOFTWARE\WOW6432NODE\GFI\ME14\Config]
GFI MailEssentials version 12(x86): - [HKEY_LOCAL_MACHINE\SOFTWARE\GFI\ME12\Config]
GFI MailEssentials version 12(x64): -[HKEY_LOCAL_MACHINE\SOFTWARE\WOW6432NODE\GFI\ME12\Config]
GFI MailEssentials version 11: - [HKEY_LOCAL_MACHINE\SOFTWARE\GFI\ME11\Config]
GFI MailEssentials version 10: - [HKEY_LOCAL_MACHINE\SOFTWARE\GFI\ME10\Config]
GFI MailEssentials version 9: - [HKEY_LOCAL_MACHINE\SOFTWARE\GFI Fax & Voice\ME9\Config]

A dword value named ‘debug’ should be found in this location. If the value does not exist a new DWORD value called ‘debug’ should be created manually.

Change the ‘debug’ dword value as follows:
To disable debug logging, change the ‘debug’ dword value to ‘0’
To enable debug logging, change the ‘debug’ dword value to ‘1’

Once complete restart the IISAdmin Services and GFI Mail Essentials Services. Then you can delete the actual log files.

Tuesday 27 July 2010

ISAlog.bak and ISALog.bin

If you have a server with ISA Server 2004 installed you will find in the C: partition the files ISAlog.bak and ISALog.bin

These files will be approx 400MB in size and often taking up valuable disk space on an older system.

The files are not necessary and are for Microsoft PSS to use if you have a case open with them. You can remove these files by editing the registry key

HKLM\software\microsoft\isaTrace\BootTracing

Set this value to 0 and reboot the server, you can then delete the files manually and reclaim the disk space.

Thursday 22 July 2010

How to monitor for when an application terminates

I have had an application that needs to run on a SBS 2008 Server as an interactive user, the application runs from a mapped network drive and then runs from the system tray.

If the application is running it works fine, but it will on occasion terminate with no error, this means that emails do not get sent from the clients workstations and it can be hours before it is noted that the application is not running.

So I decided I needed to know if this application closes or terminates, I could then restart the application automatically and know that my users can still send emails.

I started by looking to see if I could run the application as a service so I did not need the server logged on with an interactive user, I used the SRVANY.EXE application from the Windows 2003 Resource Kit and then created a service using SC.EXE and set this to run my application as local system and interactive with the desktop.

This did run the service but the application did not load into the desktop and as a result did not work so I decided this was not the solution.

Next I looked for an application that would monitor a process and then alert when this occurred, I found Application Monitor here



This allows you to specify the application you want to monitor, how often it checks to see if it is running and then will restart the application if it detects that it is not running. I quickly setup the EXE as a check and tested by closing the application manually and the Application Monitor restarted the application!

There is no event log monitoring which would have been nice for our Managed Services application that could have notified our helpdesk, but it does work for a quick fix!

Monday 19 July 2010

Snap shot delete stops at 95% on ESXi Server

I had taken a snapshot of a server pre Windows Updates and had not deleted this snapshot for four days, when I came to delete the snapshot I could see that the process had jumped to 95% and then stayed at this level for some considerable time.

This is by design, the snapshot is a differencing file and has to write the changes back to the VMDK file and so this takes some time to complete. Don't worry in this situation, just wait for the process to complete and this may take several hours!

Tuesday 13 July 2010

Missing Toolbar in SharePoint Services 3.0

I had a customer today report that the Calendar List they had in Sharepoint 3.0 had suddenly gone missing. I looked at the site and I could see when I selected the Calendar list there was indeed no content or menu bar options for New, Action or Settings.

My first thought was that the List was probably still in Sharepoint but it had somehow got hidden, so I looked into the way to edit the page and find if the list had been hidden.

This is achieved but using the Site Actions, Edit Page option in the top right, you can then choose to Add a Web Part. At the bottom of the web parts list is the "Advanced Web Part gallery and options" link, choose this and a side bar opens and you are shown the option for Closed Web Parts.

In this Closed Web Parts was the Calendar List I was looking for, I added this back to the page and saved the changes and the Calendar was back in place.

Monday 12 July 2010

Message rejected as spam by Content Filtering

We use a cloud based email security scanning service called GFI Max Mail Protection and on the whole this is a well priced and effective service. But recently my MD noticed that he no longer received his daily digest emails from the service.

I looked at the log files and I could see that his digest emails were being dropped with a NDR in the log files of 550.5.7.1 Message rejected as spam by Content Filtering.

I placed a support call as I couldn't think why his emails would be dropped as we use this is a our sole email scanning service, but it was a busy day and I didn't give it as much attention as I should have.

I get a email back from the GFI helpdesk to say, its my server that is dropping the emails, they checked their logs and could see my Exchange 2007 Server making the drop. A quick think and of course Content Filtering is part of Exchange 2007 Anti Spam features and naturally it was switched on!

This feature uses an algorithm to filter emails and it was seeing the digests as spam, probably because the digest contains HTML links to the emails so you can filter them manually. I quick switch off of this filter and my digests are being delivered again, which goes to show you should always have a good think on an issue before you fire off an email to the helpdesk!

Windows Disk Clean Up and SQL Database files

We had an issue today where users could not rollover a client in an application that used SQL Server Express 2005. The errors in the event log were listed as event ID 5118 and had the content.

The file "C:\Program Files\Microsoft SQL Server\MSSQL.1\MSSQL\Data\MyDatabase.mdf" is compressed but does not reside in a read-only database or filegroup. The file must be decompressed.

The problem had been caused by the Disk Clean Up tool being used on this server and the option to compress old files had been used, this resulted in the MDF and LDF files being compressed and this error occurs.

I entered the properties for the files after stopping the SQLEXPRESS Service and then in the Advanced button in the file properties I chose to uncompress the files. Once I had restarted the SQLEXPRESS Service the database was available to use again.

Sunday 11 July 2010

Acronis - Failed to read sector 63 of hard disk

I have used the Acronis True Image and Backup & Recovery applications to backup customers servers to USB disk, this makes for a quick, cheap and effective backup for the smaller SMB Servers we have.

On occasion we would try to run a disk backup and hit the error "failed to read sector 63 of hard disk xx" where xx is one of the hard disks in the computer.

Now if you Google this error you will find a lot of issues with this error related to Acronis True Image and that in Backup & Recovery this has been fixed in build #11639. Well I have this fault today and I am running build #11639 so I don't think this stands true yet.

I found that when I tried to backup my server with disks C: and D: to USB disk on disk F: this error would be displayed. The event log would show an event ID 1 with the details

Error Code:500
Failed to read from disk
Failed to read from sector 63 of hard disk '1'

The hard disk 1 in this server was a dynamic disk, no RAID just a Ultra 320 SCSI disk. I had checked the disk and I new this was good using CHKDSK.

I looked back in the application event log and noted event ID 12289, this was with the Source VSS. This lead me to think the error may be occuring with the VSS snapshot and the event ID details confirmed this.

Volume Shadow Copy Service error: Unexpected error OpenService (shSCManager, 'VSS', SERVICE_QUERY_STATUS). hr = 0x80070005.

This error shows that an problem is occurring when Acronis tries to take the Volume Shadow Copy Service snapshot and this then leads to the sector 63 error. A further event ID 1 then displayed this information about the snapshot

Error code: 502
Operation with partition '0-0' was terminated.
Details:
Read error.

Error code: 0x70003
Tag: 0x2CBDD167CBCA9516
Failed to read the snapshot.

Error code: 0x10C45A
Tag: 0x14181C22EF45AD6E
Access is denied

If Access is Denied then this could be a permissions issue, I looked at the backup account I was using in Acronis and this was a purpose backup account that was a member of the Backup Operators Group. By default the Backup Operators Group does not have permission to the Volume Shadow Copy Service and therefore cannot read or write the shadow copies and will display the error.

So I added my backup account to the local Administrators Group, this was acceptable in this situation as I controlled the backup account and needed to get a backup completed!

I re ran the job and this time it has ran through successfully with the VSS snapshot taken as expected.

I think the conclusion is that the "sector 63" error can be caused by a multiple of things and no one solution will fit and a methodical approach to trouble shooting is the best plan of action.

Automate a backup with SQL Server 2005 Express

When it comes to backing up a SQL database it is simple to initially setup a SQL Maintenance Plan and schedule this to backup the database and log files and then start to work on a comprehensive strategy.

But the problem comes when you need to do this with a SQL Server 2005 Express database as this version does not come with a SQL Agent so you cannot use a Maintenance Plan.

The solution comes in using a SQL Script file, configure this to backup the databases and then move these files off to a location to be backed up to tape or disk.

The first thing to do is to install the SQL Server Management Studio as this will help you generate the SQL script file, the Management Studio can be downloaded here



Once you have connected to the SQL Server instance, find the database you want to backup and then choose the option to backup the database. Once you have setup the backup choose the option to Script | Script Action to File.

This will allow you to save the backup and a .SQL file which can then be ran as a script later. Save the file to a location for your SQL backup scripts.

Now you have a .SQL file you can open this and Management Studio will open the file and hows the SQL commands, execute the script to test the backup. If this is successful and you have the relevant .BAK files you are ready to automate this process.

Using Windows Scheduled Tasks create a new task to backup the SQL Databases and when asked for the application to use browse to

C:\program files\microsoft sql server\90\tools\binn\SQLCMD.EXE

This application will run the .SQL file as a command line, next save the task with the appropriate schedule for your backups.

Open the task and edit the command line for the task to add the details of which instance to backup and where the .SQL file is located. This is appended to the command as follows

SQLCMD -S .\SQLSERVERINSTANCE -i "C:\MySQLBackup\Backup.sql"

Save the task and manually run the task to confirm it will backup your databases correctly.

Once this part is completed you can now setup the move of the BAK files with the robocopy application.

Robocopy is a great tool to copy and move files, and it is part of the Windows 2003 Resource Kit that can be downloaded from

http://www.microsoft.com/downloads/details.aspx?familyid=9d467a69-57ff-4ae7-96ee-b18c4790cffd&displaylang=en

Robocopy has many options but in my case I wanted to move the files to another location for backup to disk so I used the command

robocopy C:\MySQLBackups \\server\sqlbackup /MOV

The /MOV switch will delete the source files once copied but will NOT delete the source folder, this means we keep the folder referenced in the SQL backup script.

I then used scheduled tasks again to create a new task that executes the robocopy command at 30 minutes after the SQL backup executes.

Now I have a daily SQL backup and the files are moved off to a network share for backup to disk later.

Saturday 10 July 2010

Application Log with constant Perflib Errors

I had a server that I could not get a verification on the backup because the application log was full of Perflib Event ID 1008 errors. One was being generated about every 5 seconds so the log was full and the backup application, Acronis Backup & Recovery 10.0 could write to the log but I would never see it with my Managed Services application because the application log was being over written constantly.

So to resolve this problem, and this means stopping the error being generated in the log and not stopping the error itself, I downloaded the Windows 2000 Resource Kit tool ExCtrlList.exe from Microsoft. This is all outlined in this KB article

http://support.microsoft.com/kb/299059

Next I used this application to find the Perflib monitor that was filling my application log, in this case it was ASPNET 2.0 with the Open command, then I deselected the check box to enable the Performance Monitor.

Back to the application event log and the constant event id 1008 has stopped and I can read the event log normally.