How to lookup a user’s current Active-Directory site

Posted on March 13, 2013

Curious what site you're in?

nltest /dsgetsite

Windows Domain DFS namespace – access is denied using domain FQDN, access allowed using server UNC paths directly

Posted on November 5, 2012

This was easily one of the most frustrating and ridiculous fun times I've had working with DFS.

The issue: At several client locations we run file server redundancy by offering (2) DFSR servers. A shared domain namespace with replicated folders to ensure they stay online if a server is offline for planned or unplanned good times. Within group-policy, we map folder redirection to a namespace path:

  • "documents" -> "\\domain.com\users\username\documents"
  • "desktop" -> "\\domain.com\users\username\desktop"
  • ......

By referencing the namespace, it will redirect when server A or B is offline. This should NOT be used in WAN deployments, LAN is fast and therefore replication is fast. Initially the DFS issue was identified when drives mapped to the namespace were missing. Within the client event logs, we saw "access denied" errors associated with these drive-letters.

What we checked and verified:

  • Problematic client stations could not connect to "\\domain.com\dfsroot"  (access denied)
  • Problematic client stations could not connect to "\\domain\dfsroot" (access denied)
  • Problematic client stations could connect to "\\serverA\dfsroot"
  • Problematic client stations could connect to "\\serverB\dfsroot"
  • Permissions on the shares for the DFS Root folder were correctly set to "everyone" with read/write
  • Each of these systems was removed and rejoined to the domain [no success]
  • The local profiles were completely removed from the local systems (file system and registry) and logged back in [no success]
  • Security suites were removed [no success]
  • Each user was tested on working machines and had no issues obtaining the right drives

The culprit: 

  • When we disabled the 'offline files' component and rebooted -> "\\domain.com\dfsroot" was immediately accessible
We ultimately came to this conclusion: 

The offline file cache was corrupt. When offline files are disabled, the system accesses the namespace location directly without issue. This confirms a reference to the namespace is clearly saved within offline file cache. If the cache is corrupt you end up with "Access is Denied". Another quick way to determine if the issue is corrupt cache is to simply try and access the DFS root UNC paths on each server. If you can browse the contents when bypassing the shared namespace path, and this user has no issues on other domain PCs, then it's not permissions.
...........
The Fix:
..........
1) Disable offline files

Control Panel -> Sync Center -> Manage Offline Files -> Disable Offline Files

2) Clear the offline file cache

This sets a temporary registry entry which is read on start-up and runs the cache wipe.
Elevated Command Prompt -> "reg add HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Services\Csc\Parameters /v FormatDatabase /t REG_DWORD /d 1 /f "

3) Reboot

You must reboot to successfully wipe the offline file cache

4) Test the namespace path -> "\\domain.com\dfsroot"

If you can now browse the namespace contents, you can optionally re-enable Offline Files.
We understand the Offline Files component is critical to road warriors. You should be safe to re-enable it and reboot.

Control Panel -> Sync Center -> Manage Offline Files -> Enable Offline Files -> Reboot

After you log back in, check that you can still access the namespace path "\\domain.com\dfsroot" after you run a forced sync. If there are still issues, I recommend you follow the steps we initially took "What we checked and verified:" and repeat this fix.

Good luck!

Windows backed up failed with following error code ‘2155348129’ – Hyper-V VSS Writer – [5] Waiting For Completion – Unexpected Error

Posted on October 15, 2012

We've run into several instances where this error presents itself on Windows 2008R2 servers running Hyper-V. When you see this error in the event log around the time your backup fails, it will look similar to:

Event ID: 521
The backup operation that started at '‎2012‎-‎10‎-‎15T01:44:31.444000000Z' has failed because the Volume Shadow Copy Service operation to create a shadow copy of the volumes being backed up failed with following error code '2155348129'. Please review the event details for a solution, and then rerun the backup operation once the issue is resolved.

There are numerous issues that will cause this error. The most interesting was relating to virtual machine drive-space. In our case, we found a few virtual machines with little or no free drive space were the cause.

First determine if the issue is related to the Hyper-V Writer:

1) Load a command prompt
2) vssadmin list writers

look for the:

Writer name: 'Microsoft Hyper-V VSS Writer'
Writer Id: {66841cd4-6ded-4f4b-8f17-fd23f8ddc3de}
Writer Instance Id: {c73cdd59-f1d2-40be-b1b4-0c11449528a3}
State: [1] Stable
Last error: Unexpected Error

** Definitely related to the Hyper-V VSS writer ***

3) net stop vmms 
4) net start vmms
5) Check that the "last error:" has cleared itself:

vssadmin list writers

look for the:

Writer name: 'Microsoft Hyper-V VSS Writer'
Writer Id: {66841cd4-6ded-4f4b-8f17-fd23f8ddc3de}
Writer Instance Id: {c73cdd59-f1d2-40be-b1b4-0c11449528a3}
State: [1] Stable
Last error: No Error

6) Ensure you have at least 15% free diskspace on all Hyper-V Virtual Machines drives. 
7) Re-run your backup and monitor

If it repeats itself, I suggest looking at alternative solutions found online including resetting folder permissions.

Why is w3wp.exe CPU utilization is high – Exchange is slow – Active-Sync 2010 and iOS diagnostic tools

Posted on August 28, 2012

Microsoft is aware and working on a known issue relating to iOS devices causing high CPU utilization on Exchange Servers. The exact cause seems to bounce all over the place, in general is related to Active-Sync and how the iPhone communicates with the Exchange Server. The issue is challenging. Their direct recommendation is to ensure all devices are up-to-date. A single remote active-sync device can each up 90+% of your system resources.

The good news is there are now scripts available to help you isolate the specific device(s) causing the issue.

I suggest you read this article thoroughly:
http://blogs.technet.com/b/exchange/archive/2012/01/31/a-script-to-troubleshoot-issues-with-exchange-activesync.aspx

To make life a bit easier, use the following steps.

1) If you find that your CPU utilization is rammed, verify that you see the following process causing the issue:

By default some of these columns are not shown.
To load them, head to the "View -> Select Columns"  

Make sure both items in RED match your own process list.
If they are not identical, you are not dealing with the same issue.

2) Download the PowersShell Script: http://gallery.technet.microsoft.com/scriptcenter/ActiveSyncReport-script-a2417a84

Save the script with your other exchange scripts located under "c:\program files\microsoft\exchange servers\v14\scripts" 

3) Download and Install the LogParser: http://www.microsoft.com/download/en/details.aspx?displaylang=en&id=24659

4) Load the Exchange Management Shell and change directory to "c:\program files\microsoft\exchange servers\v14\scripts" 

Now that you're in the scripts folder ([PS] C:\Program Files\Microsoft\Exchange Server\V14\Scripts>), we can start hunting down the source of your issue. Before we do, you should be aware that the script you are about to run can take 2-5 minutes to complete (depending on the size of your IIS logs). It will count the number of 'hits' a device has sent to your server. Anything over 1000 hits is high, anything over 1500 is very high.

Let's start by checking EVERY log against every device. This should give us a general idea of those who have excessively high hits over a long period of time. The following line will save the report to "C:\EASReports", the minimum hits we are looking for is => 1000, and the result should be saved to and HTML viewable report.

.\ActiveSyncReport.ps1 -IISLog "C:\inetpub\logs\LogFiles\W3SVC1" -LogparserExec "C:\Program Files (x86)\Log Parser 2.2\LogParser.exe" -ActiveSyncOutputFolder c:\EASReports -Minimum Hits 1000 -HTMLReport

Building Log Parser Query...

Found time-taken in the IIS Log, adding this column.
Gathering Statistical data
Running Log Parser Command against the IIS Log(s): C:\inetpub\logs\LogFiles\W3SVC1\*.log

Statistics:
-----------
Elements processed: 24083868
Elements output: 1342
Execution time: 402.31 seconds (00:06:42.31)

Generating the Minimum Hits Report.
Building Log Parser Query...
Running Log Parser Command against the CSV results to determine Minimum hits of 1000

Statistics:
-----------
Elements processed: 1342
Elements output: 566
Execution time: 0.02 seconds

LogParser Command finished CSV, File location: c:\EASReports\EASyncOutputReport-Multiple_Files_Minimum_Hits_of_1000.csv
Creating HTML Output...
HTML File location: c:\EASReports\EASyncOutputReport-Multiple_Files_Minimum_Hits_of_1000.html

 If you open this log for review (c:\EASReports\EASyncOutputReport-Multiple_Files_Minimum_Hits_of_1000.html) you will see something similar to:

Take note to the DeviceID and Hits columns. In this particular example, I see 5 very devices that are clearly communicating with the exchange server excessively. Using the device ID, we can drill down further into that specific device and find out how many hits per hour. Make sure to use the DeviceID from the table above in the line below.

.\ActiveSyncReport.ps1 -IISLog "C:\inetpub\logs\LogFiles\W3SVC1" -LogparserExec "C:\Program Files (x86)\Log Parser 2.2\LogParser.exe" -ActiveSyncOutputFolder c:\EASReports -deviceID <DEVICEIDHERE> -hourly -htmlreport
Building Log Parser Query...
Found time-taken in the IIS Log, adding this column.
Gathering Statistical data for device: <DEVICEID>
On a per hourly basis.
Running Log Parser Command against the IIS Log(s): C:\inetpub\logs\LogFiles\W3SVC1\*.log

Statistics:
-----------
Elements processed: 24087512
Elements output: 3083
Execution time: 154.74 seconds (00:02:34.74)

LogParser Command finished CSV, File location: c:\EASReports\EASyncOutputReport-Multiple_Files_Hourly_<DEVICEID>.csv
Creating HTML Output...
HTML File location: c:\EASReports\EASyncOutputReport-Multiple_Files_Hourly_<DEVICEID>.html

If you open this log for review (c:\EASReports\EASyncOutputReport-Multiple_Files_Hourly_<DEVICEID>.html) you will see something similar to:

That's a lot of hits! This post was made on August 28th, and we can see on the 26th they were clearing 6000+ hits per day from this device. At this point, you should contact the user and update the iOS device to the latest version. After it's updated, watch it closely. Don't forget to check the other 5 devices on this list with high hits. It could be more than one device causing the issue.

5) The odd exception 

The first report we ran includes hits from all logs. What about recently added devices (say 1 week ago) that haven't had time to register huge numbers? For example, in the first report we saw 100,000+ hits on the first 6 devices. What if we added a new device 1 week ago that was registering 5,000 hits per day? It would only show up as 30,000-40,000 hits.

While checking the highest hits is always a good idea, you should also check the last few days individually. The example below includes a date variable (8-28-2012), modify as necessary.

.\ActiveSyncReport.ps1 -IISLog "C:\inetpub\logs\LogFiles\W3SVC1" -LogparserExec "C:\Program Files (x86)\Log Parser 2.2\LogParser.exe" -ActiveSyncOutputFolder c:\EASReports -MinimumHits 1000 -date 8-28-2012 -HTMLReport

The resulting report will show you devices on the specified date that have hit over 1000 hits. This will help you isolate a more accurate daily breakdown.

 

Happy hunting!

Active Directory Login failure on Blackberry Administration Server (BAS) – The username, password, or domain is not correct. Please correct the entry.

Posted on July 24, 2012

Logging into BAS can fail if DNS records are incorrect or stale.
Before we get into that, I highly recommend you look through your BAS logs to analyse the error.

1) Try to login to the BAS admin service using your LDAP/Active-Directory credentials several times. This will register a few errors in your logs that you can track down.
2) Head to: C:\Program Files (x86)\Research In Motion\BlackBerry Enterprise Server\Logs\(TODAYS DATE)\
3) Open the log file that starts with "SERVERNAME_BBAS-AS-" (Make sure it's the most recently modified version!)
4) Scroll to the bottom and look for lines that start with "(XX/XX XX:XX:XX:XXX):{http-"

(XX/XX XX:XX:XX:XXX):{http-URL.COM%2F10.100.1.36-3443-5} [com.rim.bes.basplugin.activedirectory.LDAPSearch] [INFO] [ADAU-1001] {u=SystemUser, t=30847} performPagedLDAPSearch problem performing LDAP operation: url=ldap://ldapserver.domain.com:3268 base= filter=(&(objectClass=user)(objectCategory=person)(|(sAMAccountName=besadmin)(userPrincipalName=besadmin))) scope=2

If you see a message similar to this, BAS is trying to grab a Kerberos ticket and your DNS is causing errors.

Resolution:

1) Load your Active-Directory DNS management console.
2) Verify that you have reverse DNS setup for the entire domain. You require PTR records (reverse dns records) for each of your domain controllers. If you don't have them, FIX THIS!
3) Verify that you have no stale records pointed to decommissioned or retired domain controllers. Drill down into each DNS folder and confirm the hostname and IP match your current infrastructure. It's amazing how many stale records we find.
4) Once repairs are made to the DNS settings, right click the server name in the DNS management and "clear cache"

Reboot your BES server just for fun, and try to login.

HP System Management HomePage – Processor none – No Hardware Listed

Posted on July 15, 2012

If you recently installed the HP System Management Homepage on a new or existing HP server and you're missing the actual content when you login, you could be missing a quick community setting on your SNMP Service.

1) Goto start -> run/search -> Services.smc
2) Scroll down the list and find "SNMP Service" -> Right Click -> Properties -> Security Tab
3)  Under "Accepted Community Names" look for a "Public" community name. If it's missing, you need to add it. Click "add". Choose "ReadOnly" and enter community name "public".
4) Click Add and then OK to close the windows.

You should now have this:

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

5) Right click the "SNMP Service" and choose "Restart". This will require it's HP dependencies to restart as well, click ok and let it finish

Log back into the HSM Homepage and you should now see everything:

Fortinet SSL VPN Authentication with LDAP – no_matching_policy

Posted on July 12, 2012

We identified a new bug with the Fortinet authentication mechanism being used to query LDAP. If you are using active-directory to authenticate remote SSL-VPN users with no success, please check your event log for a "no_matching_policy" error. Recent escalations with Fortinet have confirmed the username and password being used for LDAP queries must not include complex characters.

Our testing confirms that using characters like "#", "!", and "$" in the password can create the "no_matching_policy" issue. After removing them and sticking to alpha-numeric only, the authentication immediately works. This bug is not applicable to the incoming SSL-VPN credentials; it ONLY affects the account used to query LDAP.

Things to know about the LDAP Server settings:

 

1) Common Name Identifier

Using the "sAMAccountName" as shown above will authenticate your users with their assigned active-directory username. This is case-sensitive.

 

 

 

 

 

 

 

 

2) Distinguished Name

This is the LDAP path to your highest level query folder. You can increase security by drilling down into a specific Organizational Unit (OU), if you prefer. The authentication mechanism will only allow users within or below this path/folder/OU.

Example: DC=domain,DC=CA (this would search the full domain)
Example: OU=users,DC=domain,DC=ca (this would search objects within the users OU only)

3)  User DN (Distinguished Name)

The Fortinet appliance needs a valid and working active-directory user to run queries. This account is used to test incoming SSL-VPN login credentials against your domain controller. You MUST use the complete LDAP path  to this user object. Failure to include everything, even the spacing, will break things. I suggest you look into DSQUERY or ADSIEDIT.MSC for copy/paste of the path.

Example:  CN=LDAPUser,OU=Users,DC=domain,DC=ca

4) Password (for your User DN)

The password field on this page is reserved for the User DN (the user object you will use to send LDAP queries to your domain controller). To avoid the bug described at top of this article, use ALPHA-NUMERIC CHARACTERS ONLY! If you use "!", "#", "$", etc. you may run into failure. I also suggest you do not make your password exceptionally long. Watch your event log for the "NO_MATCHING_POLICY" issue and tweak accordingly.

 

Testing SSL-VPN Authentication from the CLI 

Fortinet has a nice built-in diag you can run to test things. Pick a user with SSL-VPN access and run the following command:

diag test authserver ldap "server-name" aduser "some.user" "password" 

 

Adding 64bit print drivers to 32bit Windows Servers

Posted on May 18, 2012

There has been a lot of confusion on this topic, so let's clear it up!

If you intend on sharing your printer from a 32-bit server to 64-bit clients, you will need to ensure both 32 and 64-bit drivers are installed at the server. The issue however is that the server will not allow you to install 64-bit drivers.  There is a really quick and simple solution.

1) Download the 32 and 64-bit Universal Printer Drivers from the manufacturer's site. DO NOT try and fiddle around with anything else.

2) Log into an as a domain administrator on a 64-bit PC or Server located in the same domain

3) From the command prompt run "printmanagement.msc" 

---- Right Click "Print Servers" -> "Add/Remove Servers"
---- Enter the hostname of your server and click "Add to List"
---- Click "Apply" and "OK"

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

---- Expand the server you entered and load the "Drivers" child object
---- Right click the centre pane and  choose "Add Driver"
---- Choose "x64" and "x86"
---- Complete the driver installation by assisting the installer to the INF files located in your 32 and 64-bit driver downloads you already completed
---- If you successfully installed the drivers, you should see both 32 and 64-bit listed under the drivers pane

4)  It's now time to install and share the printer!

Browse to "Printer Management Console" ->  "Printer Servers" -> "Server Name" -> "Printers"  -> right click "Add Printer"

 Complete the printer installation and select the drivers you installed when it asks.

5) Check the driver options 

Select the printer you installed -> right click "properties" -> "Sharing" -> "Additional Drivers"

You should see both x64 and x86 drivers checked off; you're ready to roll!

If you do not see both checked off:

Select the printer you installed -> right click "properties" -> "advanced tab"
Choose the correct driver from the drop down list under "Driver: " and recheck the additional drivers options to verify.

 

 

Port Forwarding through an IPSEC tunnel to a remote server/pc with Fortinet

Posted on March 1, 2012

Typically this situation should never happen, you can always forward ports through at the other side. Right? As all you IT guys know, 3rd party software and complex networks will ask for some weird deployment tasks. This demonstration assumes you already have an existing Interface based IPSEC tunnel running between 2 fortinet appliances and they are talking like best friends. I've tested this on 4.0 MR3 P5, I would suggest you use the same or current.

SiteA
Fortinet A

--- WAN1: 55.55.55.20
--- Int1: 10.20.1.1
--- Port 36008 should be forwarded to 10.10.1.17 in Site B

SiteB
Fortinet B

--- WAN1: 66.66.66.60
--- Int1: 10.10.1.1
--- Server on 10.10.1.17 with port 36008 open and ready

IPSEC Interface Name: IPSEC-YKN1-SPG1

The idea ... when anyone on the internet connects to 55.55.55.20 on port 36008, they will be redirected up through the IPSEC tunnel and to the remote server.  Let's get started!

1) FortinetA - Create a port-forwarding rule just like any other

 

2) Fortinet A - Create a firewall policy to allow the internet traffic in on WAN1 and out the IPSEC Interface to your port forwarding rule (remote server)

Do not forget to enable NAT!!!

3) Fortinet B - Create a reverse route to manage the WAN1 interace on Fortinet A

Without this, the fortinet will drop the incoming packets when it does a reverse path check.

"id=36871 trace_id=188 func=ip_route_input_slow line=1268 msg="reverse path check fail, drop"

I recommend you set the distance to 1 so that it takes priority, you should decide depending on your other routes.

Instead of using the /24, you could use the gateway the WAN1 port of the Fortinet A is using. For this demonstration to help keep your sanity, we've left it at the full subnet.

4) Test it out! 

The best method to test this is to telnet to port 36008 from an EXTERNAL location to 55.55.55.20.
You can do this with any operating system.

Telnet 55.55.55.20 36008

If it connects to your service, you're good to go!

If you're having issues, I suggest changing the forwarding port to a port that you know is open on the server. Try using telnet from the inside of your network to verify the port is open, and then test externally.

Common server ports are 21, 22, 80, 110, 143, 3389

Check which ports are open on your server, head back to Step 1) and use this port instead.
Once you have it routing through to that port correctly, change it to the actual port you need.

 

 

 

Move or transfer Windows 2008/2008R2 virtual machines between vSphere/ESXi hosts using Windows Backup

Posted on March 1, 2012

Something to note, this will work regardless of your ESXi versions. Because we are relying on the windows backup instead of vMotion or other tools, the VM host/datastore is not relevant.

1) Create a network share

This share must reside on a server or PC accessible to you virtual machine, and the user you will be used to connect to it must have full/read.

2) Log onto your existing virtual machine

Verify you can access this network share by IP or hostname (ie. \\computername\sharefolder). Do not proceed further until you can see and write to this folder from 1). 

3) Install the Windows Server Backup tools 

Load the Server Manager > Features > Add Features > Windows Server Backup Features (everything) > install 

4) Load the Windows Server Backup tools

Load the Administrator Tools -> Windows Server Backup 

5) Create a backup and dump your image to the network share you created 

Windows Server Backup > Backup Once > Full Server (recommended) > Remote Shared Folder > \\computername\sharefolder > Inherit 

You will be prompted for credentials.
MAKE SURE you use credentials that have full read/write access to the folder your shared!

START the backup! 

6) While the backup is running, create an empty VM on the new vSphere host

Load up your vSphere client ->CTRL-N

Configuration: Typical
Name:  You tell me!
Storage:  Choose the location for the new VM
Guest OS: Windows 2008 R2 64bit

Network:  Pick 1 network card, the correct network, and E1000 with Power On. DO NOT CHOOSE ANOTHER TYPE OF NETWORK CARD!
Windows 2008 R2 will recognize the e1000 with it's native drivers. You can change the network settings later.

Disk Provisioning: The size of your virtual disk size should be AT LEAST the size of the old VM virtual disk size.
I should note that you can also use this opportunity to INCREASE the size of your partitions. If your old VM is running out of space, crank it up here and Windows will increase the size of your partitions on restore.

Finish: Choose to "edit the virtual machine settings before completion" and click continue.

7) Verify your VM settings 

Make sure the RAM, CPU and other settings are verified for your old VM. This would be an excellent time to beef it up if you need to.

8) Configure the CD/DVD to mount and boot off of a Windows 2008/2008R2 Server image

Click the "New CD/DVD (adding)" devices -> side window -> Datastore ISO File or Host Device or Client Device > Find the ISO or DVD for windows server 2008 / 2008R2
Enable the "Connect at power on" option at the top.

Finish.

9) Check the status of the running backup from earlier. Do not proceed until the backup is completed and successful. 

Once the backup is complete, power down the old VM. If you turn it back on, you run the risk of IP address, DNS and hostname conflicts.

10) Start the new Virtual Machine and start the Windows 2008 installation process

Choose your default language, time, keyboard -> next
Choose "Repair your computer" (bottom left)
Choose "restore your computer using a system image that you created earlier" > next
Choose "cancel" when it reports "Windows cannot find a system image on this computer"
Choose "select a system image" > next
Choose "advanced" > "Search for a system image on the network" > "yes"

You will now be provided a field to enter the network path to the backup share we created earlier. Make sure to type it in exactly as your testing value.

\\computername\sharefolder

Click OK (it will now attempt to access that folder)
Enter the login credentials for the share you created.

If everything you entered and shared is correctly setup, you will be provided a populated table with the backup completed earlier.



 

 

 

 

 

 

 

 

 

 

Choose Next, Next

11) If you wish to increase the size of the partitions from the old VM, this is your chance.

12) Finish it up and get the restore started

This process could take minutes to hours depending on the size of your old VM.

13) Finish the new VM configuration and take it live

At this point, the snapshot of your previous VM is now live on the new VM.
The last steps include:

- Install the VMware tools using the built-in tools
- Adjust any network settings , network card types (i prefer the VMXNET3 drivers myself)
- Double check all services and applications are live after you reboot (vmware tools and network changes should be followed by a reboot)