Avantgarde Technologies

<a href="http://www.avantgardetechnologies.com.au">Avantgarde Technologies</a>
Perth's IT Experts

Thursday, May 21, 2015

SSTP VPN Bug with Server 2012 R2

Tonight I stumbled across what appears to be a bug/glitch with Server 2012 R2 when attempting to setup an SSTP VPN connection.  I completed the config as per MS best practice with a valid public digital certificate which I obtained from DigiCert however when I went to connect, I received the following error:

Error Code: 0x80072746
Error Description: 0x80072746: An existing connection was forcibly closed by the remote host.


My certificate was bound to the RRAS instance correctly and TCP443 was forwarded through.

 
When I ran a "netsh http show ssl" however it did not show the binding, despite me configuring it in the RRAS interface and the certificate existing in the local computer certificate store with private key.
 
 
I tried changing the certificate in RRAS to another, then changing it back, but still no binding.
 
As a result, I exported the certificate off the machine with Private Key, deleted the certificate, then imported it back with Private Key.  After performing this task, I then rebound it in Routing and Remote Access (RRAS) using the same method.
 
Running the command again, the binding worked.
 
 
 And that's why we get paid the big bucks!

Wednesday, May 20, 2015

Testing GRE Protocol with PPTP VPN connections

In this post I am going to show you how to test GRE Protocol is open and working when setting up PPTP VPN Servers.  GRE Protocol being blocked on network infrastructure is one of the most common issues faced when setting up a PPTP VPN Server.

The first step is to stop the VPN Server.  This is done by right clicking the VPN server in Routing and Remote Access and clicking "Stop" under All Tasks.  This is done because our testing tool also listens on TCP1723 so we need to ensure this port is no longer in use on the server in question.
 
 
Next download our testing tools called pptpsrv.exe and pptpclnt.exe.  These tools are available in the Windows XP SP2 support tools available from the following URL:
 
 
Copy the pptpsrv.exe file to C:\Windows\System32 on your PPTP VPN Server then run it from an Administrative Command Prompt.  Ensure Windows Firewall does not block it.  Run the tool on the server and it will begin listening on TCP1723.
 
 
 
Next run the pptpclnt.exe from another remote computer and target it against the PPTP VPN Server like this:
 
pptpclnt.exe 192.168.2.11
 
My local IP address of my PPTP VPN Server is 192.168.2.11.  Then when prompted enter in some sample text to send through the TCP1723 port socket.  I wrote "Hello this is clint".
 
 
Next go back to the PPTP VPN Server and verify the text was received.  We can see that the VPN Server received the text "Hello this is clint" and then went onto the second test of closing down the port socket TCP1723 and beginning a socket for GRE Protocol test.

 
The client automatically goes on to test GRE protocol.  GRE Protocol on my local network is currently blocked by our Cisco Infrastructure (L3 Switch/Router) and as a result we get a Socket Failure error.
 
 
This means I need to go back to the Cisco Infrastructure, resolve this problem then test again.
 
Hope this post has been helpful for troubleshooting PPTP VPN servers.

Wednesday, May 6, 2015

Shared Mailboxes and Sent Items in Microsoft Exchange

In this post I'm going to focus on an Exchange feature introduced back Exchange 2010 SP2 Update Rollup 4 which has not had heavy exposure out there in the real world however is a feature in demand by many companies.  Ever since Exchange 5.5, companies have been able to create shared mailboxes which multiple users use to access email content.  Users with shared mailbox are able to view incoming email which is handy for many shared mailbox scenarios such an "Accounts mailbox", a "Service Desk mailbox", a "HR mailbox" and so on.  However, when a user with access to the shared mailbox "Sends As" the shared mailbox or "Sends on behalf" of the shared mailbox, the Sent Item goes to the users mailbox, not the shared mailbox.  This means users that share the shared mailbox cannot see the email sent on behalf of the mailbox.

See the problem?

Well this has been addressed in Exchange 2010 SP2 Update Rollup 4 and the latest update rollup in Exchange 2010 SP3.  If you are still on Exchange 2003 or 2007 you will need to upgrade to be able to use this feature.

What Microsoft has done is implement a new feature which "copies" the email "Sent As" or "Sent on behalf" of the shared mailbox TO the shared mailboxes sent items from the users primary mailbox.  This means the sent item will appear in both the shared mailbox and the users mailbox.

This needs to be manually configured on each shared mailbox with the following command:

Set-MailboxSentItemsConfiguration "shared mailbox name" -SendAsItemsCopiedTo SenderAndFrom -SendOnBehalfOfItemsCopiedTo SenderAndFrom

This command configures both SendOnBehalfOf and SendAs, if you only want to configure one of them, simply remove the one you want to not configure.

There is a problem however, this command the Set-MailboxSentItemsConfiguration and Get-MailboxSentItemsConfiguration commands were not included with Exchange 2013 or Office 365 and are still not available in the latest release Exchange 2013 CU8.

However, this feature will be coming out soon in Exchange 2013 CU9 but will have different commands and will no longer use the Set-MailboxSentItemConfiguration and Get-MailboxSentItemConfiguration cmdlets.  The commands instead will use the Set-Mailbox cmdlet with the following syntax:

set-mailbox -MessageCopyForSentAsEnabled $False
set-mailbox -MessageCopyForSendOnBehalfEnabled $False
set-mailbox -MessageCopyForSentAsEnabled $True
set-mailbox -MessageCopyForSendOnBehalfEnabled $True

Also, this feature will be enabled BY DEFAULT on all mailboxes in Exchange 2013 CU9.  Companies will no longer need to worry, shared mailboxes will automatically receive any emails "Sent As" or "Sent on behalf as".

More information on this new feature in Exchange 2013 CU9 can be found below:

http://blogs.technet.com/b/exchange/archive/2015/03/03/want-more-control-over-sent-items-when-using-shared-mailboxes.aspx

Monday, April 6, 2015

Unable to Remove Mailbox Permission in Exchange 2013

After a cross-forest domain  migration, a customer complained that a number of their users have random SIDs of unknown accounts linked to their mailbox as shown in the following screenshot.


 This object is not inherited anywhere in the configuration partition in Active Directory, I checked the ACLs on the Databases Exchange Org and the Active Directory containers in between.

The following error shows the mailbox permission in question in more detail showing it is not Inherited.  It also shows the error message we receive when attempting to remove it:

WARNING: Can't remove the access control entry on the object "CN=ceo,OU=Civic Centre,OU=CEO,OU=Office of CEO,OU=Users,OU=COMPANY,DC=DOMAIN,DC=LOCAL" for account "S-1-5-21-3350901170-1262693169-1119774923-3651" because the ACE doesn't exist on the object.


The same error is also experienced if we include "-InheritanceType All" on the command.


After reading into this a bit more, some people have posted that using the Exchange 2003 management tools allows the object to be removed.

Source:

https://social.technet.microsoft.com/Forums/en-US/50a94a45-903e-409e-ba5c-116d84bed7ff/cannt-remove-full-access-rights-for-one-user?forum=exchangesvrdeploylegacy

I do not believe building a WinXP machine and installing the legacy Adminpak.msi and Exchange 2003 system manager tools is the way to go!

After scratching my head for a while, it turns out that adding "-Deny:$True" to the end of the command allowed me to remove the ACL.

 
Hope this post saves you some time figuring this one out!
 

Monday, March 30, 2015

How to Remove SID History from Active Directory Object

In this blog post I'm going to show you how to remove the SIDHistory from an object in Active Directory after a domain migration.  If you attempt to use standard Microsoft tools such as ADSIEdit to remove the SIDHistory from an object regardless what access rights you have been assigned, the following error will be presented.

Operation failed.  Error code: 0x5
Access is denied


00000005: SecErr: DSID-031A1256, problem 4003
(INSUFF_ACCESS_RIGHTS), data 0




To remove SIDHistory from an object you need to use the following VBScript from Microsoft KB295758.

http://support.microsoft.com/en-us/kb/295758

Simply copy and paste the script into a notepad document then run the script with the following arguments to remove the SIDHistory entries from the object in question.

 

Sunday, March 29, 2015

Windows 8 Unable to Connect through RD Gateway

A customer of mine today logged a support case stating users on Windows 8 or Windows 8.1 were unable to connect to remote computers by using a Remote Desktop Gateway (RD Gateway).

I tested this functionality and was able to reproduce the issue.  The error experienced was:

Remote Desktop can't connect to the remote computer for one of these reasons:
  1. Remote access to the server is not enabled
  2. The remote computer is turned off
  3. The remote computer is not available on the network
Make sure the remote computer is turned on and connected to the network, and that remote access is enabled.


Windows 7 clients did not receive any problems connecting through an RD Gateway.

After reviewing the group policy applied to the domain, I noticed a Group Policy object was setting the LAN Manager Authentication Level to "Send LM & NTLM - use NTLMv2 session security if negotiated".

 
The new RDP 8.0 client (built into Windows 8 and 8.1) requires this policy be set to "Send NLTMv2 response only" as it uses NTLMv2 and does not have the ability to negotiate authentication.  "Send NTLMv2 response only" is the default for Windows 8 and Windows 8.1.
 
As a test, I set a local policy on one of the Windows 8 computers using gpedit.msc and then did a gpupdate /force followed by a reboot.
 
 

After setting the LAN Manager authentication level to "Send NLTMv2 response only" I was able to connect to RD Gateways without issues.

I did not test this, but I assume if a Windows 7 client was updated to RDP v8 by installing Microsoft KB2592687, the same issue would be experienced if the LAN Manager authentication level is changed to anything other then the default.

Friday, March 27, 2015

PowerShell - Find All Files Beginning With

A customer of mine was hit with another one of those Viruses which encrypt all data on shared drives mapping back to the file server.  The entire shared drive was encrypted and users were no longer able to access documents on the volume.

I restored all encrypted files from backup however I still had these HELP_DECRYPT Ransome ware files in every directory on the file server.


As a result I needed an easy way to find and delete each of these files.

PowerShell!

First set the path you want to search, mine was H:\Shared.

Next run the following command to search any files containing HELP_DECRYPT with the following command:

Get-ChildItem $Path -Recurse | Where{$_.Name -Match "HELP_DECRYPT"}


 This went through and listed all of these HELP_DECRYPT files in every directory of the file server recursively.

After you have carefully went through all the results and confirmed that no legitimate files were listed, you can pipe the output from the Get-ChildItem command into Remove-Item cmdlet.


After piping the Output into Remove-Item, run the command to list the items again to ensure they were all deleted correctly.  Getting no output as per above means the files were removed successfully.
 

Saturday, March 21, 2015

Repairing ContentIndexState on DAG Nodes in Exchange 2013

In Exchange Server 2013, sometimes the content index state can go corrupt on databases.  When this occurs, Exchange 2013 FAST search technology is no longer available for the database meaning people cannot search for content from OWA, Active Sync or Outlook in online mode.

In DAG environments, you can simply reseed the Content Index State from a healthy node in the cluster.  I will show you how to perform this in this blog post.

Here we have two databases in a DAG cluster which have the index status in a failed state:


To manually update the ContentIndexState from a healthy node simply run the following command:

Update-MailboxDatabaseCopy "database\server" -CatalogOnly

In my case the database I want to update was "CCEX2-DB-02" on server "CCEX1" so I ran:

Update-MailboxDatabaseCopy "CCEX2-DB-02\CCEX1" -CatalogOnly


After running the command you can see we brought the index for the database back to a healthy state.


Repeat this process for any other databases with a failed content index.

Friday, March 20, 2015

Public Folder Migration Error:Property expression isn't valid

When migrating public folders from legacy Exchange 2007 and Exchange 2010 environments to Exchange 2013, you may receive the following error message:

Error: Property expression "Anglicare RT" isn't valid. Valid values are: Strings formed with characters from A to Z (uppercase or lowercase), digits from 0 to 9, !, #, $, %, &, ', *, +, -, /, =, ?, ^, _, `, {, |, } or ~. One or more periods may be embedded in an alias, but each period should be preceded and followed by at least one of the other characters. Unicode characters from U+00A1 to U+00FF are also valid in an alias, but they will be mapped to a best-fit US-ASCII string in the e-mail address, which is generated from such an alias.


This is caused by an invalid alias format generally created from legacy versions of Exchange such as 2000 or 2003.  Exchange 2010/2013 does not support spaces in the Public Folder Alias.

On the legacy Exchange 2007/2010 server open up Public Folder Management console and navigate to the Public Folder in question.  On the Exchange General tab you will receive an error message saying the object contains invalid data.


Simply remove the space from the Alias, this is no longer supported.

 
Next go back to your Exchange 2013 server and resume your public folder migration request.
 
 
You may need to repeat this process a few times as it is likely multiple public folders have incorrect aliases.

Monday, February 23, 2015

VSS Snapshot error. The maximum number of snapshots for this volume has been reached.

Today on a customers SBS 2008 server still running Backup Exec 2010, backups started failing with the following error.

 - AOFO: Initialization failure on: "\\JCC-SBS\Microsoft Information Store\First Storage Group". Advanced Open File Option used: Microsoft Volume Shadow Copy Service (VSS).

V-79-10000-11234 - VSS Snapshot error. The maximum number of snapshots for this volume has been reached.  Microsoft Volume Shadow Copy Service (VSS)  will not allow any more snapshots.

The Advanced Open File Option was set to Automatically select open file technology - I changed this to use "System - Use Microsoft Software Shadow Copy Provider".


After making this change, I tested another backup and it completed successfully.


Interestingly, if you read the error it was already using the Microsoft Volume Shadow Provider yet it failed.  Hardcoding it in the settings of the backup job did the trick.

Phantom Calls to Cisco Phone with Hosted VOIP Service - How to Lockdown

An issue was experienced where a number of VOIP phones on a customer network were experiencing phantom calls from the Internet.  A phantom call generally occurs when someone is scanning for SIP devices on a network using a port scanning program.  A port scan against SIP (usually UDP 5060 and 5061) will cause many SIP handsets to annoyingly ring and when the user answers, no one is there!

As you could gather, this is very annoying for users on the network.

The solution for this is locking down SIP to only the Public IP addresses of your SIP IP Gateway from your VOIP provider on the public interface of the router.  This customer was using a Cisco 887va router connected on a DSL connection so the public interface in this instance is dialer0.

First of all you need to create a access rule.  Here is the Access Rule I created (this matches the Faktortel VOIP providers public IP addresses here in Australia).  Faktortel also use 5062, 5063, 5064 and 5065 UDP ports for SIP communication.

ip access-list extended VOIPLockDown
remark allow SIP from certain addresses
permit udp host 202.43.66.55 any eq 5060
permit udp host 202.43.66.55 any eq 5061
permit udp host 202.43.66.55 any eq 5062
permit udp host 202.43.66.55 any eq 5063
permit udp host 202.43.66.55 any eq 5064
permit udp host 202.43.66.55 any eq 5065
permit udp host 202.43.66.1 any eq 5060
permit udp host 202.43.66.1 any eq 5061
permit udp host 202.43.66.1 any eq 5062
permit udp host 202.43.66.1 any eq 5063
permit udp host 202.43.66.1 any eq 5064
permit udp host 202.43.66.1 any eq 5065
permit udp host 202.43.66.2 any eq 5060
permit udp host 202.43.66.2 any eq 5061
permit udp host 202.43.66.2 any eq 5062
permit udp host 202.43.66.2 any eq 5063
permit udp host 202.43.66.2 any eq 5064
permit udp host 202.43.66.2 any eq 5065
permit udp host 202.43.66.3 any eq 5060
permit udp host 202.43.66.3 any eq 5061
permit udp host 202.43.66.3 any eq 5062
permit udp host 202.43.66.3 any eq 5063
permit udp host 202.43.66.3 any eq 5064
permit udp host 202.43.66.3 any eq 5065
permit udp host 202.43.66.4 any eq 5060
permit udp host 202.43.66.4 any eq 5061
permit udp host 202.43.66.4 any eq 5062
permit udp host 202.43.66.4 any eq 5063
permit udp host 202.43.66.4 any eq 5064
permit udp host 202.43.66.4 any eq 5065
permit udp host 202.43.66.5 any eq 5060
permit udp host 202.43.66.5 any eq 5061
permit udp host 202.43.66.5 any eq 5062
permit udp host 202.43.66.5 any eq 5063
permit udp host 202.43.66.5 any eq 5064
permit udp host 202.43.66.5 any eq 5065
remark
remark block SIP from all other addresses
deny udp any any eq 5060
deny udp any any eq 5061
deny udp any any eq 5062
deny udp any any eq 5063
deny udp any any eq 5064
deny udp any any eq 5065
remark
remark Allow all other IP traffic
permit ip any any


This rule permits all SIP traffic to all of their public IP addresses then blocks all other traffic entering on these ports and finally permits any/any.

Next you need to assign the access rule to the public interface on the router you wish to filter.  Because this is a Cisco 887va router on DSL, the public interface is Dialer0.  Simply apply the access rule to Dialer0 as follows:

interface dialer0
ip access-group VOIPLockDown in


After applying this config change, all phantom calls to phones on the network stopped.

Important: Some SIP providers use both TCP and UDP for SIP, so it can be useful to block both UDP and TCP on these port numbers.  For this customer, the SIP handsets only supported UDP so there was no point blocking the TCP ports.  If your not sure, block both by slightly modifying the config above.

Tuesday, February 3, 2015

Direct Access Clients Cannot Establish Network Connectivity after NLS Becomes Unavailable

In Microsoft Direct Access Deployments, you need to configure a server (or server cluster) on your network to act as a Network Location Server (NLS).  The NLS is simply a Web Server with a HTTPS server certificate... see the following TechNet article on how to configure an NLS:

https://technet.microsoft.com/en-us/library/ee649252(v=ws.10).aspx

If the DirectAccess client can establish a connection to the NLS, it will believe that it is on the internal network and the DirectAccess tunnels will not be established. If a connection to the NLS cannot be established, the DirectAccess client believes it is outside the corporate network and will attempt to establish DirectAccess connectivity. It is for this reason that the NLS should not be resolvable in public DNS and should not be reachable externally. If it is, DirectAccess clients will always think they are inside the corporate network, and DirectAccess will never be established. Also, it is extremely important that the NLS be highly available. If the NLS server is offline or unreachable for any reason, DirectAccess clients that are on the internal network will suddenly think they are outside the corporate network and attempt to establish DirectAccess connectivity. This will fail, leaving DirectAccess clients unable to connect to any internal resources until the NLS is once again available.  For this reason it is strongly recommended that you implement at least two NLS servers in a cluster, using either Network Load Balancing (NLB) or an external hardware load balancer.

However for smaller deployments of Direct Access, a NLS cluster for redundancy is not always practical and sometimes only a single NLS server is deployed.  This is often the case with all in one Direct Access Deployments running Windows Server 2012 R2 when a single Direct Access server is deployed to act as both the NLS and Direct Access endpoint.  If the all in one Direct Access server was to fail, all clients on the Internal network will no longer think they are on the internal network as they cannot contact the NLS and as a result attempt to establish a Direct Access connection which will also fail leaving them in a state of no connectivity to any domain names configured with a Name Resolution Policy Table (NRPT) in the Direct Access group policy.  Most Direct Access deployments have the Active Directory domain namespace configured as an NRPT policy and sometimes additional domain names.  To understand what NRPT is, please see the following blog post:

http://clintboessen.blogspot.com.au/2014/01/what-is-name-resolution-policy-table.html

What Happens if Your In this Situation?

What happens if your in this situation when you have a single Direct Access server which has failed and as a result all Direct Access configured clients on the internal network now have no network connectivity?

The NRPT Policies which are pushed out via Group Policy are located under the following registry key:

HKEY_LOCAL_MACHINE\SOFTWARE\Policies\Microsoft\Windows NT\DnsClient\DnsPolicyConfig

You can manually connect to the direct access computers and remove these registry keys by IP address using the "Connect Network Registry" option in regedit.exe.  Once you are connected to the remote computer, you can delete the two NRPT policies named DA-{GUID].


After you have deleted the two policies remotely, simply reboot the workstation and it will regain connectivity.

I know this is a very manual process and can be very time consuming if there are hundreds of computers experiencing the issue - I would like to hear from Microsoft for a better solution in this scenario.  I sent of an email regarding this and hopefully will hear back soon!

It is also important to note, for my customer who's Direct Access server failed, rebuilding a new server with the same IP address, hostname and configuration did not resolve this issue.  I still needed to delete the NRPT policies from the registry from the effected computers and reboot.  After this, the clients had restored connectivity to the internal network and were able to refresh group policy again in which they downloaded the new Direct Access Client Settings containing the new NRPT policies.

Friday, January 23, 2015

Test Application for Group Policy Software Installation Troubleshooting

Sometimes when diagnosing problems with Group Policy software deployment, it is good to try a different application to rule out issues with the application package your trying to deploy.  One application package I have found which is great for testing Group Policy software deployment is "RealWorld Paint".  This is a lightweight MSI installer (8.7 MB) in size and deploys easy through Group Policy to Windows XP, Windows 7 and Windows 8.1 machines.

You can download this MSI installer from the following location:

http://download.rw-designer.com/2013.1/RWPaint32.msi

If you have issues deploying this with Group Policy, then the issue does not lay with the MSI package your trying to deploy but something else!

Also check out this common issue with Group Policy application deployment, it may be of help in diagnosing your Group Policy deployment issue:

http://clintboessen.blogspot.com.au/2013/01/group-policy-software-installation-not.html

Tuesday, January 20, 2015

HyperV Network Types

Microsoft HyperV servers have three Network Types you can assign to Virtual Machines and Virtual Switches.
  • Private
  • Internal
  • External
This is shown below on both a virtual machine and a virtual switch configuration.



 
 
What is the difference between these?
 
An External Network Type provides communication between a virtual machine and a physical network by creating an association with a physical network adapter on the virtualization server.  This is the most common type used by organisations.
 
An Internal Network provides communication between the virtualization server and the virtual machines.
 
A Private Network only provides communication between the virtual machines and not the HyperV host server.
 
Internal and Private network types can be confusing - the only difference is Internal also allows the virtual machines to communicate with the HyperV host server.

Thursday, January 8, 2015

GFI MailEssentials 2014 SR2 Transport Issues during Exchange Migration

I had an issue with a third party email filtering product GFI MailEssentials 2014 SR2 during an Exchange 2010 to Exchange 2013 migration.  GFI MailEssentials 2014 SR2 is a spam filtering product which you install directly on an Exchange transport server.  It integrates with Exchange server through six transport agents which all perform various tasks as shown below:

 
When migrating to a new version of Exchange, as part of the process you are required to redirect mail flow to the new Exchange transport server so it can route the mail to the legacy transport environment until a point when the mailboxes can be moved.
 
As the new Exchange 2013 server will be the new external point of connectivity for SMTP, I installed GFI Mail Essentials on the new Exchange 2013 server and redirected mail flow as shown below.
 
 
After making this change, users were not able to receive email from external users.  I verified the following things:
  • Exchange 2013 was receiving emails from external users as validated in the SMTP Protocol logs.
  • Exchange 2013 was forwarding emails to the Exchange 2010 server as per standard functionality.
  • Exchange 2010 successfully received the email communication from Exchange 2013 at transport level and was verified in the protocol logs.
  • GFI MailEssentials Transport Agents on the Exchange 2010 server receive the email for processing.
  • GFI MailEssentials does not place the email back into the Exchange Pickup folder giving it back to Exchange for processing.
I was not able to locate where the emails were moved to within GFI primarily due to my limited knowledge of the product (to me it is just a custom Exchange transport agent).  I contacted GFI support here in Australia who were also unable to advise me where the emails went after being relayed to the Exchange 2010 server.  Fantastic, so we have emails going into a black hole disappearing forever.
 
One thing GFI support were able to advise me was their transport agents only filter email which was sent from a public IP address, all private IP addresses were excluded from filtering.  This was in line with my symptoms, all users internally were able to receive emails sent from internal devices such as Printers being relayed through the Exchange 2013 server.
 
In the following screenshot I have included the message tracking log from the Exchange 2010 server.  The first two entries are from when the Cisco Router was forwarding email directly at the Exchange 2010 server.  All other entries are from when mail was relaying through Exchange 2013.  As you see email is received via SMTP however not delivered to the information store via the Exchange Store Driver due to GFI not releasing the mail.
 
 
GFI Mail Essentials modifies the header of emails that are filtered and appears to not deal with emails correctly which already have their header modified by another instance of GFI Mail Essentials.
 
As a work around I simply disabled the GFI Transport Agents on the Exchange 2010 server to prevent it from interfering with mail processing to resolve the issue as shown below:
 
 
This resolved the issue and did not compromise the environment as security and spam filtering was now being performed by GFI on the Exchange 2013 server.

Wednesday, January 7, 2015

Microsoft Exchange Virus Scanning API (VSAPI) Removed

Back at the MVP Summit 2012 in Redmond, Microsoft announced to the Exchange MVP community that in Exchange 2013 they are going to pull the Microsoft Exchange Virus Scanning API (VSAPI) from the product.  This API is what allows anti-virus products to scan inside the information store for emails.  This early news came to me with a big smile on my face!

For years I have been advising customers NOT to install anti-virus products which scan the information store as it causes unnecessary load on the information store and has caused database corruption at some of my customers.  Despite my advice, some of my clients go ahead and installed this functionality anyway to meet a "compliance" checkbox which some integrator has flagged in a security audit.

I have always advised customers to perform anti-virus scanning at a transport level (SMTP) and flag emails before they reach the database to improve performance and allow for greater scalability.  It is important to note however, Anti-Virus products are always releasing new definitions and it is possible that a virus was allowed in due to the definitions not being able to detect it initially but being able to detect it at a later date.  Hence, this still proposes a risk to the business and can be caught using third party Anti-Virus products which use the Microsoft Exchange Virus Scanning API (VSAPI) right?  Well yes this is true, however I still do not recommend this.  A better solution is to run cached Exchange mode and allow client side Anti-Virus products to scan the users offline cache "OST file" for viruses on a regular basis and offset this load from your already busy mail servers.  This approach meets the same objective and does not require use of the Microsoft Exchange Virus Scanning API.

One of my customers who went against my advise and refused to disable Information Store scanning due to compliance requirements on Exchange 2010 now has no option but to remove it.   Microsoft Support must have finally had enough of dealing with issues from third party Anti-Virus products causing information store issues just like me!

Have a look at the following screenshot comparing a product GFI MailEssentials 2014 SR2 which has the ability to scan the information store on a Exchange 2010 server compared to an Exchange 2013 server.  Under the Email Security menu on the Exchange 2013 server (on the right), you will se the feature gone for good... not just in GFI but all anti-virus products.


A big thankyou to Microsoft for removing this API - this is one less argument I need to have with my customers!

Lastly, I still always recommend companies AppLock with Microsot AppLocker in Enterprise edition of Windows and do away with definition scanning Anti-Virus solutions, they are a thing of the past.  This is another argument I'm still having with the security compliance guys stuck in the dark ages, but we will save this for another blog post.

Sunday, January 4, 2015

RemoteApp Disconnected This computer can't connect to the remote computer

After deploying a Remote Desktop Services environment on Server 2012 R2, users complained of an issue where they were no longer able to launch RemoteApps from the RD Web App portal.  The error they received was:

RemoteApp Disconnected

This computer can't connect to the remote computer.

Try connecting again. If the problem continues, contact the owner of the remote computer or your network administrator.


To resolve this issue open the properties for your application collection experiencing problems, navigate to security and untick "Allow connections only from computers running Remote Desktop with Network Level Authentication".

 
This sorted out the problem in my environment.

Wednesday, December 24, 2014

Exchange 2013 FAST Search Technology Failed

I had a customer where Exchange 2013 FAST Search Technology failed and rebuilding the search indexes for the databases did not resolve the problem on a single Exchange 2013 SP1 server.

In this blog post I'm going to share with you the symptoms of my issue along with the resolution.

Symptoms of Issues

Users on the network were unable to search their mailbox from Outlook Web App or using Microsoft Outlook in online mode.

In addition to not being able to search in Outlook Web App or Microsoft Outlook, the Test-ExchangeSearch cmdlet failed with "Time out for test thread".


Despite this issue, the ContentIndexState on the databases remained in a Healthy report status with the Get-MailboxDatabaseCopyStatus cmdlet as shown below:


In the Server Application logs the following error was logged numerous times:

Log Name:      Application
Source:        MSExchangeIS
Date:          4/12/2014 1:54:06 PM
Event ID:      1012
Task Category: General
Level:         Error
Keywords:      Classic
User:          N/A
Computer:      ExchangeServer.domain.local
Description:
Exchange Server Information Store has encountered an error while executing a full-text index query ("eDiscovery search query execution on database a0976351-948a-4625-8840-f649f8b98e0e failed."). Error information: System.ServiceModel.EndpointNotFoundException: Could not connect to net.tcp://localhost:3847/. The connection attempt lasted for a time span of 00:00:02.0821592. TCP error code 10061: No connection could be made because the target machine actively refused it 127.0.0.1:3847.  ---> System.Net.Sockets.SocketException: No connection could be made because the target machine actively refused it 127.0.0.1:3847


Checking the size of the catalog folder was only around 16MB when the database size was 150GB.  From my experience a FAST index for a 150GB database is generally between 5GB and 10GB in size depending on how small the items are inside the database.

Issue Resolution

To resolve the issue the following steps were taken in order.

1. Stop the Microsoft Exchange Search and Microsoft Exchange Search Host Controller service.
 
2. Remove all the files under “Fsis” file:

C:\Program Files\Microsoft\Exchange Server\V15\Bin\Search\Ceres\HostController\Data\Nodes\Fsis
We moved these files to another folder for backup purposes.

3. Open EMS and “Run as Administrator"

4. Navigate to directory C:\Program Files\Microsoft\Exchange Server\V15\Bin\Search\Ceres\Installer

5. Launch the script and run the command below:


.\installconfig.ps1 -action I -datafolder "c:\Program Files\Microsoft\Exchange Server\V15\Bin\Search\Ceres\HostController\Data"
This script will recreate the files under "C:\Program Files\Microsoft\Exchange Server\V15\Bin\Search\Ceres\HostController\Data\Nodes\Fsis" that we backed up/removed previously.

6. Once the script installs the data, you need to uninstall and reinstall again (don't ask, it did not work for me without doing this):

.\installconfig.ps1 -action U -datafolder "c:\Program Files\Microsoft\Exchange Server\V15\Bin\Search\Ceres\HostController\Data"

Then again installed Search component with below command :

.\installconfig.ps1 -action I -datafolder "c:\Program Files\Microsoft\Exchange Server\V15\Bin\Search\Ceres\HostController\Data"


7. Force Active Directory Replication from a domain controller in the same AD Site as the Exchange 2013 server with repadmin /syncall /APeD.

8. Check on the Exchange Server with Get-MailboxDatabaseCopyStatus and review the ContentIndexState.


When the Content Index process has finished crawling we can see the index size of the folder is now 8GB and searching is now working as normal.

 

Tuesday, December 23, 2014

Filtering Exchange MessageTrackingLog Output from Get-MessageTrackingLog Cmdlet

In todays blog I am going to show you how to filter Message Tracking Log output from the Get-MessageTrackingLog command.  I had a customer from an organisation example.com who wanted to see all external recipients who received an email with the message subject "Staffing Update".

Without using some PowerShell filtering, you can view all tracking log related entries for this against all Transport Servers in your organisation by running:

Get-TransportService | Get-MessageTrackingLog -MessageSubject "Staffing Update"

Note: If your running Exchange 2007 or 2010 replace Get-TransportService with Get-TransportServer.

This command will return all messages relayed in the last 30 days by default which is the amount of time the message tracking logs hang around for by default.

However if we want to meet my customers requirement, we want the output to not contain any recipients for "*@example.com" so that we can focus on emails leaving the company.  When I say external users or "leaving the company", I mean users who do not have an email address for "*@example.com", you may have an Exchange Organisation where you have multiple accepted domains setup for multi-tenancy!

To do this we need to do some filtering with PowerShell.  This can be achieved using the where{ } command as follows:

Get-TransportService | Get-MessageTrackingLog -MessageSubject "Staffing Update" | where{$_.recipients -notlike "*@example.com"}

But say we only want to see which users received this message with a @example.com email address.  Easy this can be done by reverting the -notlike to a -like.

Get-TransportService | Get-MessageTrackingLog -MessageSubject "Staffing Update" | where{$_.recipients -like "*@example.com"}

You can also easily filter senders by replacing the $_.recipients line with $_.senders like shown below:

where{$_.sender -like "*@example.com"}

Happy Filtering!

Wednesday, December 10, 2014

Microsoft PST Capture 2.0 Not Supported on Exchange 2013

The PST Capture 2.0 tool is not supported with Exchange 2013 despite being documented as supported under the following Microsoft Exchange Product Team blog post:

http://blogs.technet.com/b/exchange/archive/2013/02/22/time-to-go-pst-hunting-with-the-new-pst-capture-2-0.aspx

I have used this tool successfully with previous versions of Exchange Server however with Exchange Server 2013 this tool does not work.  I have tested PST Capture 2.0 with Exchange Server 2013 SP1 and Update Rollup 6, both unsuccessful.  Testing has been performed both in my lab environment and at a customer site.

I logged this issue with Microsoft under a professional services support case in which Microsoft were able to reproduce the issue in their lab environment.  This issue was then escalated within Microsoft and an issue identified with the tool with no fix provided.  Microsoft explained they are under no obligation to develop a hotfix for this tool as the End User Licensing Agreement (EULA) states that the software is provided "as is" and they are not under obligation to support it should issues or bugs be flagged with the software.  This is shown in my screenshot below.


When attempting to import a mailbox with PST Capture 2.0, the following error message is flagged in the tool:

Import error: Error opening mailbox


This issue is generally caused by one of the following issues as documented on the following website:

http://www.sysadminsblog.com/microsoft/pst-capture-errors/

However with Exchange 2013, none of these resolve the problem.

The PST Capture tool has a more advanced logging available for errors which can be found under the following location:

"C:\ProgramData\Microsoft\Exchange\PST Capture\Logs\MapiComServer 15"


The technical error which is generated when attempting to import to Exchange 2013 is as follows:

Failed to open default store - HR 8004011d


Hopefully Microsoft does get around to developing a fix/work around for this product as it will be a shame to purchase this tool from Red Gate Software and then not maintain it hence making it redundant.

Wednesday, December 3, 2014

How to Change Network Profile in Registry for 2008 Server

In a previous post I wrote about how to change the network location or "firewall profile" in Windows Server 2012 using PowerShell as you could no longer do this from the GUI like you could in previous versions of Windows Server. This article can be found under the following URL:

http://clintboessen.blogspot.com.au/2013/11/how-to-change-your-network-profile-in.html

However yesterday I had a problem at a customer running Windows Server 2008 where I was unable to change the network location from the GUI using Network and Sharing Centre and as a result I had to revert to the registry.  Tracking down the registry key responsible for the network profile was a difficult task so I decided to blog this one.

All network profiles are located under the following registry key:

HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\Windows NT\CurrentVersion\NetworkList\Profiles


Each network card will be represented below in a sub key with a GUID.  The Network Location is controlled by a DWORD value called Category.

Public = 0
Home = 1
Domain = 2

As you see my category is set to 2 which means this network adapter is using the Domain Firewall profile.

Tuesday, December 2, 2014

Remove Power Shell and Server Manager Pinned Icons from Start Menu Server 2012

In Windows Server 2012 and Windows Server 2012 R2 by default all users have a Power Shell icon and a Server Manager icon pinned to their start menu.  When setting up a Remote Desktop Session Host with Windows Server 2012 Remote Desktop Services you may not want users to have the Server Manager icon or Power Shell icon especially when desktop access is enabled.

 
In previous versions of Windows Server there was a Group Policy setting called "Remove pinned programs list from the Start Menu" which administrators could use to remove these pinned applications from the start menu from Remote Desktop Session Hosts.
 
 
This policy no longer works on Windows Server 2012 and there is no well documented method available for doing this on the Internet.
 
In Windows Server 2012, the three default pinned applications on the start menu are located under the following location:
 
C:\Users\username\AppData\Roaming\Microsoft\Internet Explorer\Quick Launch\User Pinned\TaskBar
 
 
The User Pinned\Taskbar  folder in the Quick Launch directory does not exist in the Default profile in Windows Server 2012 as shown below.


So how do these icons get created in the users profile and where do they come from?

This process appears to be hardcoded into Windows Server 2012 and executed upon creating a new user profile.  The user profile creation process creates Quick Launch\User Pinned\TaskBar folder as part of the login then copies a bunch of "lnk" shortcut files from the "All Users Profile".

Server Manager Pinned Icon

The "Server Manager.lnk" file gets copied by the user profile creation From:

"C:\ProgramData\Microsoft\Windows\Start Menu\Programs\Administrative Tools\Server Manager.lnk"

To:

"C:\Users\username\AppData\Roaming\Microsoft\Internet Explorer\Quick Launch\User Pinned\TaskBar\Server Manager.lnk"

Windows PowerShell Pinned Icon

The "Windows PowerShell.lnk" file gets copied by the user profile creation From:

"C:\ProgramData\Microsoft\Windows\Start Menu\Programs\System Tools\Windows PowerShell.lnk"

To:

"C:\Users\username\AppData\Roaming\Microsoft\Internet Explorer\Quick Launch\User Pinned\TaskBar\Windows PowerShell.lnk"

How to Prevent the icons from getting Populated to User Profiles

To stop the "Windows PowerShell.lnk" and "Server Manager.lnk" shortcuts from getting pinned to the start menu on all users, simply delete these icons from the All Users profile under "C:\ProgramData" from the following locations:

"C:\ProgramData\Microsoft\Windows\Start Menu\Programs\Administrative Tools\Server Manager.lnk"

and

"C:\ProgramData\Microsoft\Windows\Start Menu\Programs\System Tools\Windows PowerShell.lnk"

 
Now when new users login for the first time, the User Profile creation process is unable to copy the "Server Manager.lnk" and "Windows PowerShell.lnk" shortcuts as they no longer exist as shown in the following screenshot.