TN - 1283 Troubleshooting Deployment Failures

Description

This article from InSource covers a deployment 'checklist' to go through when faced with a deployment issue

• Author: Mario Meza
• Published: 09/16/2022 (Edits on 12/20/2024)
• Applies to: Application Server 2023 R2 and above

Details

1. Run Change Network Account Utility by Right Clicking and selecting “Run as Administrator” (even if you are an Administrator)   DIRECTORY:     C:\Program Files (x86)\Common Files\ArchestrA\aaAdminUser.exe

2. Run OSConfig Utility    “Run as Administrator” (even if you are an Administrator)

     a.C:\Program Files (x86)\Common Files\ArchestrA\OSConfigurationUtility.exe

3. Validate Binding Order / Interface Metric of the primary NIC  (AVEVA Technote for Network Metric)
   1. Requires an AVEVA GCS user to access Technote 

4. TURN OFF System Management Server if possible or validate that it is not being used.

     Make sure to run Configurator with the option “Run as Administrator” (even if you are an Administrator)  

5. Firewall Turned Off or Exceptions in place

                                Available in the SP Install Guide PDF on the Installation Media.

6. AV Exceptions in Place

                                 Available in the SP Install Guide PDF on the Installation Media

7. From Command prompt execute ping /a for getting through to DNS and to see what the return time is
   1. A DNS reply 5 second to DNS Server is no good. Per AVEVA you want reply 4 seconds or less for good deployment
   2. Best practice is to use a host file just need to make sure it’s updated should IP’s change.
   3. You need to ping by both host name & IP (the needs to be done from both sides of the connection)

8. Validate Host File for accuracy  (TN WW208)

 LOCATION: C:\Windows\System32\drivers\etc

10. Local Security Policy->Local Policies->Security Options ->     

               “System cryptography: Use FIPS complaint algorithms for encryption, hashing and signing” - Disabled

11. Local Security Policy -> Local Policies -> Security Options ->  

                “Network Access: Let everyone permissions apply to anonymous users - Enabled

12. Apply TN 000022811 Troubleshooting Wonderware Application Server Bootstrap Communications

13. Validate NT SERVICE\aaPIM is in Administrators Group and Service Logon are correct for the ArchestrA services

      (Ref TN 10225 New Accounts and Security Groups in Wonderware System Platform 2017 Update 3 and later )

14. Logged in user is part of aaAdministrators and aaConfigTools group

15. Ensure any Installed KB is validated against Security Central Export  (One approach is outlined Step 3 of TN 10131)          

16. Test with a new object instance (Also applies to UDOs, Engines, Areas, Platforms etc)

17. Try a deploy from another galaxy or create a test Galaxy (when possible)

18. Run a Repair of the Software (ENSURE UNBLOCKED MEDIA is USED)

NOTE:   You may encounter an orphaned platform or a hung deployment while trouble shooting. This can result in a deployment conflict / mismatch.

Here are 3 options for dealing with them.

OPTION 1: Use the OCMC (formally known as the System Management Console or SMC)

The preferred method to recover in these scenarios is to use the Remove Local Platform (highlighted in green halo box below) from the deployed client (NOT Galaxy Repository Platform (GR!)) 

Then in the System Platform IDE (formally known as the ArchestrA IDE) you would undeploy the node that was ‘Removed’ with the option ‘On fail mark as Undeployed’.

In some scenarios you may not see the 'remove local platform' option available in the OCMC.

OPTION 2: Use the Killer / Remover Utilities

There are some utilities that remove platforms when run from the from the deployed client (DO NOT RUN ON Galaxy Repository Platform (GR)!) 

Platform Remover (2016) WSP 2017 and above : https://insourcess.box.com/s/plik1oc6lgfrxbtfq0fjfv60nt82wjvt

A² Platform Killer WSP 2014 and below: https://insourcess.box.com/s/ijmjli3ddi06m0m85icbmonazvknx649

After running the appropriate utility for your version on the deployed client.

In the IDE you would undeploy the node that was ‘Removed’ with the option ‘On fail mark as Undeployed’

OPTION 3: Clean Up the Registry

Finally, there is a manual process which can be done by deleting orphaned entries from the registry.

MODIFICATIONS TO THE REGISTRY ARE DONE AT YOUR OWN RISK.

BEFORE MAKING ANY MODIFICATION ENSURE SURE YOU BACK UP AND TEST FIRST IN A SANDBOX

1. From the IDE undeploy the orphaned / hung node with the option ‘On fail mark as Undeployed’ (This should remove the node from registry of all nodes).

2. Go to RegEdit ->HKEY LOCAL MACHINE -> Software -> WOW6432Node -> Archestra -> Framework -> Platform -> PlatformNodes

3. Verify that the remote node is gone from the list. If it’s not, it needs to be manually removed.

***Depending on size of galaxy this may be big task because you would have to do this on all nodes.

***Note on GR Platform0 and Platform1 should be the GR (Galaxy Repository)

Diagnostic Manager App

Nine times out of ten the above checklist list takes care of the issue, should it persist, the Diagnostic Manager App is a utility that (starting with 2017 U3) comes pre-installed with the product at: C:\Program Files (x86)\Common Files\ArchestrA\DiagnosticManagerApp.exe

When running, there is a tab called useful tip. Once selected you can select the current issue from a drop-down list. When 'Deploy' is selected, components and relevant flags to enable will be listed. Enable these, try a deployment, and collect the log export filtered to the time of the deployment. These will be key when submitting a ticket with InSource.