A process serving application pool ‘Classic .NET AppPool’ suffered a fatal communication error with the Windows Process Activation Service

I am running an ASP.NET 3.5 web application on IIS 7 on Windows Server 2008.

I received the following error earlier this afternoon:

HTTP Error 503. The service is unavailable.

I looked in the Event logs and discovered that:

A process serving application pool ‘Classic .NET AppPool’ suffered a fatal communication error with the Windows Process Activation Service. The process id was ‘3328’. The data field contains the error number.

Here is the full details of that event:

This resulted in:

Application pool ‘Classic .NET AppPool’ is being automatically disabled due to a series of failures in the process(es) serving that application pool.

Here is the full details of that event:

Concluding in the: HTTP Error 503. The service is unavailable.

Would someone please help me to identify how this happened and what I need to do to prevent it from happening in the future.

6 Answers 6

This one is a pretty nasty one. I recommend getting the IIS Debug Diagnostics tools. You can probably alleviate the symptom by turning off «Rapid Fail Protection» in the properties of your application pool, but that will just mask the problem. What is going on is that your application is experiencing an exception so often/so repeatedly that the service shuts itself off. You’ll need to run the diagnostic tool to capture everything going in and out to find the exception that’s occuring.

This can be a very laborious task because the tool will gather gigabytes of junk to sift through. If you’re lucky, the turning off of the «Rapid Fail Protection» might allow the exception to proceed through to the browser, but my guess is that the server will just end up hanging.

If you are using NServiceBus make sure whatever account the application pool is running under has permissions to read off of the message queue.

NServiceBus runs a background task on application startup that timesout after 10 seconds, causing the w3wp process to be killed if it can’t read from the queue. After n (default of 5) w3wp crashes Rapid-Fail Protection kicks in and stops the application pool.

Its most likely caused by an infinite loop.

(I just had the same problem.) So you will just have to look for places in the code that use loops and see if removing one of them stops the error. Once you have found the culprit, then you can try to figure out whats causing it to loop indefinitely.

I just tracked down and fixed the same nasty problem. A couple things that could cause the problem:

• A Server.Transfer() loop
• Calling CurrentThread.Abort() in certain scenarios.. (Like after a Server.Transfer()!)

Fixing those things is what fixed it for us.

I had this problem, the event viewer had «The Module DLL C:\Windows\system32\RpcProxy\RpcProxy.dll failed to load. The data is the error.» This post had the steps to correct this: http://forums.iis.net/t/1154189.aspx

I had the same problem now on IIS-10 / Windows Server. The app worked well in development environment but produced errors in production server. I assume the problem may be related to some memory/heap/stack management. Publishing release version instead of debug solved the issue. More detailed: «Advanced Build Settings» / «Debug Info» /»full» -> «pdb-only».

Update: The next day, I accidentally released a debug version again. Now the running software produced a bit different behaviour. Now the solution was to disable «Check for arithmetic overflow/underflow» (Build->Advanced) (VS2013)

Not the answer you’re looking for? Browse other questions tagged asp.net http iis-7 http-status-code-503 or ask your own question.

Linked

Related

Hot Network Questions

Subscribe to RSS

To subscribe to this RSS feed, copy and paste this URL into your RSS reader.

site design / logo © 2021 Stack Exchange Inc; user contributions licensed under cc by-sa. rev 2021.4.16.39093

By clicking “Accept all cookies”, you agree Stack Exchange can store cookies on your device and disclose information in accordance with our Cookie Policy.

Fatal communication error with the windows process activation service the process id was

Вопрос

Некоторое время назад я провел миграцию c Exchange 2010 на 2016cu11 (windows 2016). Всё работало штатно, однако после установки обновления для Windows 2016 KB4493470 вместе с обновлением для Exchange KB4487563 в системных логах с периодичностью в несколько минут стала появляться ошибка:

A process serving application pool ‘MSExchangeServicesAppPool’ suffered a fatal communication error with the Windows Process Activation Service. The process id was ‘ разные числа >’. The data field contains the error number.

1. Рвутся, восстанавливаются соединения с Outlook на всех Mac,

2. В Outlook не отображаются сведения о доступности переговорных комнат (долго обновляет, после чего пишет, что не удается связаться с сервером), через owa доступность показывает без проблем.

3. Один пользователь заметил, что через owa письма в ящике не показывает вообще никакие, в Outlook же все работает.

Удаление обновления KB4487563 ничего не изменило. Сегодня ночью буду откатывать обновление для Windows.

Буду признателен за любую помощь.

Ответы

Сделал все по инструкции:

In fact, I would say that this process not only solved my issue but also got rid of a majority of the little warnings and errors on both my Exchange 2013 and Lync 2013 servers. they seem to enjoy life more when they can exchange tokens directly with each other.

For a majority of you, you will have the certificate in place, and setting up the integration will be super easy. If by chance, you do not have the cert, then this is your easy 2 min fix.

1. New-ExchangeCertificate -KeySize 2048 -PrivateKeyExportable $true -SubjectName «cn= Microsoft Exchange Server Auth Certificate» -DomainName «*.yourdomain.com» -FriendlyName «Microsoft Exchange Server Auth Certificate» -Services smtp

Do not accept to replace the SMTP certificate when prompted

2. Note the thumbprint of the new certificate. Let us assume it is 7A39541F8DF58D4821967DD8F899B27410F7C081

4. Set-AuthConfig -NewCertificateThumbprint 7A39541F8DF58D4821967DD8F899B27410F7C081 –NewCertificateEffectiveDate $a

Accept to continue despite the fact that the certificate effective date is not 48 hours into the future

5. Set-AuthConfig –PublishCertificate

6. Make sure to remove any potential reference to the previous certificate (which might not exist anymore) by doing Set-AuthConfig -ClearPreviousCertificate.

После всего этого сделал iisreset /stop /start и, на всякий случай, shutdown -r -t 0

Возможно заработало. Жду завтра.

Все ответы

1. Сделайте Recycle Pool если ещё не делали.

Установил дебаггер, в процессе разбирательства обратил внимание на ошибку в Application от ASP.NET 4.0.30319.0

An unhandled exception occurred and the process was terminated.
Application ID: /LM/W3SVC/2/ROOT/EWS
Process ID: 28880
Exception: Microsoft.Exchange.Security.OAuth.OAuthTokenRequestFailedException
Message: Missing signing certificate.

StackTrace: at Microsoft.Exchange.Security.OAuth.V1ProfileLocalTokenIssuer..ctor(ILocalConfiguration localConfiguration)
at Microsoft.Exchange.Security.OAuth.V1ProfileOAuthTokenBuilder..ctor(ILocalConfiguration localConfiguration, ITokenIssuerHelper tokenIssuerHelper, OAuthSnapshot oAuthSnapshot)
at Microsoft.Exchange.Security.OAuth.OAuthExchangeToExchangeTokenFactory..ctor()
at Microsoft.Exchange.Services.Core.GetClientAccessToken.<>c. b__49_0()
at System.Lazy`1.CreateValue()
at System.Lazy`1.LazyInitValue()
at Microsoft.Exchange.Services.Core.GetClientAccessToken.GetExchangeUserTokenForConnectors(ADUser user)
at Microsoft.Exchange.Services.Core.GetClientAccessToken.CreateConnectorsToken(ADUser user)
at Microsoft.Exchange.Services.Core.GetClientAccessToken.ExecuteGetClientAccessToken()
at Microsoft.Exchange.Services.Core.GetClientAccessToken.Execute()
at Microsoft.Exchange.Services.Core.ExceptionHandler`1.Execute(IRequestDetailsLogger logger, CreateServiceResult createServiceResult, Int32 index, ExecutionOption executionOption)
at Microsoft.Exchange.Services.Core.BaseStepServiceCommand`3.InternalExecuteStep(Boolean& isBatchStopResponse)
at Microsoft.Exchange.Services.Core.ServiceCommandBase`1. b__82_0()
at Microsoft.Exchange.Services.Core.ServiceCommandBase`1.<>c__DisplayClass88_0. b__0()
at Microsoft.Exchange.Common.IL.ILUtil.DoTryFilterCatch(Action tryDelegate, Func`2 filterDelegate, Action`1 catchDelegate)
at Microsoft.Exchange.Services.Core.ServiceDiagnostics.SendWatsonReportOnUnhandledException(ICallContext callContext, Action methodDelegate)
at Microsoft.Exchange.Services.Core.ServiceCommandBase`1.ExecuteHelper(Func`1 action)
at Microsoft.Exchange.Services.Core.Types.ServiceTask`1.<>c__DisplayClass11_0. b__0()
at Microsoft.Exchange.Common.IL.ILUtil.DoTryFilterCatch(Action tryDelegate, Func`2 filterDelegate, Action`1 catchDelegate)
at Microsoft.Exchange.Services.Core.Types.BaseServiceTask`1.SendWatsonReportOnGrayException(Action callback, Action exceptionHandlerCallback, Boolean isGrayExceptionTaskFailure)
at Microsoft.Exchange.Services.Core.Types.ServiceTask`1.ExecuteHelper(Func`1 multiStepAction)
at Microsoft.Exchange.Services.Core.Types.ServiceTask`1. b__7_0()
at Microsoft.Exchange.Diagnostics.RequestDetailsLoggerBase`1.TrackLatency[TResult](Enum latencyMetadata, Func`1 method)
at Microsoft.Exchange.Diagnostics.RequestDetailsLoggerBase`1.TrackLatency[TResult](Enum latencyMetadata, Func`1 method, Double& latencyValue)
at Microsoft.Exchange.Services.Core.Types.ServiceTask`1.InternalExecute(TimeSpan queueAndDelay, TimeSpan totalTime)
at Microsoft.Exchange.Services.Core.Types.BaseServiceTask`1.<>c__DisplayClass33_0. b__0()
at Microsoft.Exchange.Services.Core.Types.BaseServiceTask`1.LocalTimedCall(Action action)
at Microsoft.Exchange.Services.Core.Types.BaseServiceTask`1.ExecuteWithinCallContext(Action action)
at Microsoft.Exchange.Services.Core.Types.BaseServiceTask`1.Execute(TimeSpan queueAndDelayTime, TimeSpan totalTime)
at Microsoft.Exchange.Services.Core.Types.BaseServiceTask`1.ExecuteLoop(Boolean synchronously)
at System.Threading.ExecutionContext.RunInternal(ExecutionContext executionContext, ContextCallback callback, Object state, Boolean preserveSyncCtx)
at System.Threading.ExecutionContext.Run(ExecutionContext executionContext, ContextCallback callback, Object state, Boolean preserveSyncCtx)
at System.Threading.QueueUserWorkItemCallback.System.Threading.IThreadPoolWorkItem.ExecuteWorkItem()
at System.Threading.ThreadPoolWorkQueue.Dispatch()

Эта ошибка происходит в ту же секунду, в которую валится msexchangeserviceapppool. В сертификатах всё в порядке, по крайней мере через GUI. Развернул рядом еще один чистый эксчейндж cu12 на Win 16 cu5, думаю, перекидывать на него концы, если не придумаю, как пролечить правильно asp и сертификаты — надеюсь, я в правильную сторону копаю.

A process serving application pool ‘X’ suffered a fatal communication error with the Windows Process Activation Service

I’m running an ASP.NET 4.0 application under IIS 7.5. Several times a day, our application pool is recycling unexpectedly. Our system log shows the following event when this occurs:

A process serving application pool ‘X’ suffered a fatal communication error with the Windows Process Activation Service. The process id was ‘5768’. The data field contains the error number.

A process serving application pool ‘X’ failed to respond to a ping. The process id was ‘1032’.

Almost always, there is a corresponding event at the exact same time in the Application log like this:

So I installed IIS Debugging Tools and set up a crash rule, specified «A specific IIS web application pool», no logging for first chance exceptions, and nothing under «Advanced Settings» (exceptions, breakpoints, events).

I’m also (concurrently) running ADPlus from the Windows Debugging Tools with the following command line:

However, since I’ve attached the debuggers, I’ve gotten several «warning» level messages in the System log (the ones regarding the «Process serving application pool ‘X’ suffered a fatal communication error. «), but I am NOT getting the corresponding Error level event in the Application log anymore.

The only thing I’ve been able to get is the following (there were about 50 of these):

Exit code 0x800703e9 indicates a stack overflow somewhere, which is fortunate because that should be easy to fix once I can find it.

However, in order to do so, I need more information from my crash dumps so that I can open them with WinDbg and use the «!clrstack» command to pinpoint my problem.

My question is: Do I have the debugging tools configured incorrectly or am I misunderstanding the event logs? It seems our app pool is recycling whenever I see the event in the System log regarding the «fatal communication error with Windows Process Activation Service», however I’m not getting any crash dump info from IIS Debugging Tools or ADPlus when those events occur. And for some reason, since I’ve attached the debuggers, I’m not getting any more «Error» events in the Application log that correspond with my System log warning events. Not sure why. I somehow need to get the full crash dump with CLR information in it so I can figure out where my problem is.

The only other thing I can think to mention is that the Windows Error Reporting service is not running. Not sure if it needs to be or not.