Hi All,
I have generated AOS crash Dump file, from one of our Production AOS servers (Dynamics AX 2012 R3) Load balanced using NLB. I have uploaded and analysed the crash dump file, using WinDbg Tool. I would appreciate if someone could help with the following report and identify and help understand the cause of the AOS Crash please. This is the first time I am using the tool.
******************************************
Loading Dump File [C:\WER\UserDumps\Ax32Serv.exe.4100.dmp]
User Mini Dump File with Full Memory: Only application data is available
************* Path validation summary **************
Response Time (ms) Location
Deferred SRV*c:\websymbols*http://msdl.microsoft.com/download/symbols
Symbol search path is: SRV*c:\websymbols*http://msdl.microsoft.com/download/symbols
Executable search path is:
Windows 8.1 Version 9600 MP (4 procs) Free x64
Product: Server, suite: TerminalServer DataCenter SingleUserTS
6.3.9600.17031 (winblue_gdr.140221-1952)
Machine Name:
Debug session time: Fri Jan 4 10:40:42.000 2019 (UTC + 0:00)
System Uptime: 44 days 11:14:35.093
Process Uptime: 25 days 16:59:39.000
.........................................
0:025> !sym -noise
!sym <noisy/quiet - prompts/prompts off> - quiet mode - symbol prompts on
0:025> !sym -noisy
noisy mode - symbol prompts on
0:025> .reload -i ax32serv.exe
SYMSRV: BYINDEX: 0x25
c:\websymbols*http://msdl.microsoft.com/download/symbols
Ax32Serv.pdb
19B5BF9DA08E400FBA5F30BDEA8B36281
SYMSRV: PATH: c:\websymbols\Ax32Serv.pdb\19B5BF9DA08E400FBA5F30BDEA8B36281\Ax32Serv.pdb
SYMSRV: RESULT: 0x00000000
DBGHELP: Ax32Serv - public symbols
c:\websymbols\Ax32Serv.pdb\19B5BF9DA08E400FBA5F30BDEA8B36281\Ax32Serv.pdb
0:025> !analyze -v
*******************************************************************************
* *
* Exception Analysis *
* *
*******************************************************************************
SYMSRV: BYINDEX: 0x26
c:\websymbols*http://msdl.microsoft.com/download/symbols
KERNELBASE.dll
5BC006FD116000
SYMSRV: PATH: c:\websymbols\KERNELBASE.dll\5BC006FD116000\KERNELBASE.dll
SYMSRV: RESULT: 0x00000000
DBGHELP: c:\websymbols\KERNELBASE.dll\5BC006FD116000\KERNELBASE.dll - OK
Failed to request MethodData, not in JIT code range
GetUrlPageData2 (WinHttp) failed: 12002.
KEY_VALUES_STRING: 1
STACKHASH_ANALYSIS: 1
TIMELINE_ANALYSIS: 1
Timeline: !analyze.Start
Name: <blank>
Time: 2019-01-04T15:05:49.51Z
Diff: 15907051 mSec
Timeline: Dump.Current
Name: <blank>
Time: 2019-01-04T10:40:42.0Z
Diff: 0 mSec
Timeline: Process.Start
Name: <blank>
Time: 2018-12-09T17:41:03.0Z
Diff: -2073788296 mSec
Timeline: OS.Boot
Name: <blank>
Time: 2018-11-20T23:26:07.0Z
Diff: -452892296 mSec
DUMP_CLASS: 2
DUMP_QUALIFIER: 400
CONTEXT: (.ecxr)
rax=00007fff687d6500 rbx=0000000000000001 rcx=000000000000000f
rdx=00007fff687d6520 rsi=0000000000000619 rdi=0000000000000000
rip=00007fff65db8e6c rsp=0000000026bde410 rbp=0000000026bde8a0
r8=0000000000000000 r9=0000000000000000 r10=0000007800000000
r11=00007fff687d6ff0 r12=0000000000000003 r13=00000005225a9400
r14=0000000000000000 r15=0000000000000001
iopl=0 nv up ei pl nz na pe nc
cs=0033 ss=002b ds=002b es=002b fs=0053 gs=002b efl=00000202
KERNELBASE!RaiseException+0x68:
00007fff`65db8e6c 488b8c24c0000000 mov rcx,qword ptr [rsp+0C0h] ss:00000000`26bde4d0=000082cd85e8cd18
Resetting default scope
FAULTING_IP:
KERNELBASE!RaiseException+68
00007fff`65db8e6c 488b8c24c0000000 mov rcx,qword ptr [rsp+0C0h]
EXCEPTION_RECORD: (.exr -1)
ExceptionAddress: 00007fff65db8e6c (KERNELBASE!RaiseException+0x0000000000000068)
ExceptionCode: c0000005 (Access violation)
ExceptionFlags: 00000001
NumberParameters: 0
DEFAULT_BUCKET_ID: ACCESS_VIOLATION
FOLLOWUP_IP:
Ax32Serv!RaiseCriticalError+fe
00007ff7`ae9bcfce 488b8c2490010000 mov rcx,qword ptr [rsp+190h]
BUGCHECK_STR: ACCESS_VIOLATION
ERROR_CODE: (NTSTATUS) 0xc0000005 - The instruction at 0x%08lx referenced memory at 0x%08lx. The memory could not be %s.
EXCEPTION_CODE: (NTSTATUS) 0xc0000005 - The instruction at 0x%08lx referenced memory at 0x%08lx. The memory could not be %s.
EXCEPTION_CODE_STR: c0000005
WATSON_BKT_PROCSTAMP: 557276dd
WATSON_BKT_PROCVER: 6.3.2000.326
PROCESS_VER_PRODUCT: Microsoft Dynamics AX
WATSON_BKT_MODULE: KERNELBASE.dll
WATSON_BKT_MODSTAMP: 5bc006fd
WATSON_BKT_MODOFFSET: 8e6c
WATSON_BKT_MODVER: 6.3.9600.19178
MODULE_VER_PRODUCT: Microsoft® Windows® Operating System
BUILD_VERSION_STRING: 6.3.9600.17031 (winblue_gdr.140221-1952)
MODLIST_WITH_TSCHKSUM_HASH: 7597a66c7e56f39f344f9a0f1a62feaf2daf2389
MODLIST_SHA1_HASH: 421a0640ad603d49c392a50f7444a7f68244e203
NTGLOBALFLAG: 0
PROCESS_BAM_CURRENT_THROTTLED: 0
PROCESS_BAM_PREVIOUS_THROTTLED: 0
APPLICATION_VERIFIER_FLAGS: 0
PRODUCT_TYPE: 3
SUITE_MASK: 400
DUMP_FLAGS: 8000c07
DUMP_TYPE: 3
PROCESS_NAME: unknown
MISSING_CLR_SYMBOL: 0
ANALYSIS_SESSION_HOST: DEV-AXOS0007
ANALYSIS_SESSION_TIME: 01-04-2019 15:05:49.0051
ANALYSIS_VERSION: 10.0.17763.132 amd64fre
MANAGED_CODE: 1
MANAGED_ENGINE_MODULE: clr
MANAGED_ANALYSIS_PROVIDER: SOS
MANAGED_THREAD_ID: 2708
THREAD_ATTRIBUTES:
OS_LOCALE: ENG
PRIMARY_PROBLEM_CLASS: ACCESS_VIOLATION
PROBLEM_CLASSES:
ID: [0n321]
Type: [@APPLICATION_FAULT_STRING]
Class: Primary
Scope: DEFAULT_BUCKET_ID (Failure Bucket ID prefix)
BUCKET_ID
Name: Omit
Data: Add
String: [ACCESS_VIOLATION]
PID: [Unspecified]
TID: [Unspecified]
Frame: [0]
ID: [0n313]
Type: [@ACCESS_VIOLATION]
Class: Addendum
Scope: BUCKET_ID
Name: Omit
Data: Omit
PID: [Unspecified]
TID: [0x2708]
Frame: [0] : KERNELBASE!RaiseException
LAST_CONTROL_TRANSFER: from 00007ff7ae9bcfce to 00007fff65db8e6c
STACK_TEXT:
00000000`26bde410 00007ff7`ae9bcfce : 00000000`00000001 00000000`26bde8a0 00000000`00000000 00000000`00000003 : KERNELBASE!RaiseException+0x68
00000000`26bde4f0 00007ff7`ae9315eb : 00000000`26bde770 00000000`26bd0619 00000000`00000001 00000005`225a9430 : Ax32Serv!RaiseCriticalError+0xfe
00000000`26bde6a0 00007ff7`ae932462 : 00000000`59128730 00000005`223a66c4 00000000`00000003 00000000`00000001 : Ax32Serv!
CSessionMgr::CreateNewServerSession+0x30b
00000000`26bde7d0 00007fff`662c2103 : 00000000`59128730 00000005`223a66c4 00000000`00000003 00000000`00000001 : Ax32Serv!ServerGetDBSession+0x8d2
00000000`26bdebf0 00007fff`662c6ced : 00000000`26bdf1c0 00007ff7`af2ed812 00000000`59128890 00000000`00000000 : rpcrt4!Invoke+0x73
00000000`26bdeda0 00007fff`662c7dfa : 00000000`00000000 00000000`26bdf508 00000000`0cabfa00 00000000`80090301 : rpcrt4!NdrStubCall2+0x35e
00000000`26bdf410 00007fff`662c2477 : 00007fff`65ce6e20 00007fff`00000001 00000000`00000003 00007fff`653ce420 : rpcrt4!NdrServerCall2+0x1a
00000000`26bdf440 00007fff`662c22d0 : 00000000`1693a880 00000000`00000007 00000000`26bdf5f0 00000000`00000001 : rpcrt4!DispatchToStubInCNoAvrf+0x33
00000000`26bdf490 00007fff`662fa2c6 : 00000000`1693a880 00000000`00000004 0000000d`00000000 00000000`59128730 : rpcrt4!
RPC_INTERFACE::DispatchToStubWorker+0x190
00000000`26bdf590 00007fff`662fa0bb : 0000000d`4fb34dd8 0000000d`4fb34bd0 00007fff`6631e9c4 00000000`59128730 : rpcrt4!OSF_SCALL::DispatchHelper
+0x186
00000000`26bdf6b0 00007fff`6635e933 : 00000000`00000214 00000000`00000000 00000000`00000018 00000000`59128730 : rpcrt4!
OSF_SCALL::ProcessReceivedPDU+0x3ac
00000000`26bdf740 00007fff`662fc63e : 0000000d`4fb34bd0 00000000`26bdf8f0 00007fff`6631e9c4 00000000`014b9130 : rpcrt4!OSF_SCALL::BeginRpcCall
+0x123
00000000`26bdf770 00007fff`662f941c : 00007fff`662fa540 0000000d`000008cc 00000000`00000250 00000000`00000250 : rpcrt4!
OSF_SCONNECTION::ProcessReceiveComplete+0x2a4
00000000`26bdf870 00007fff`65dd4cb9 : 00007fff`662fb940 00007fff`662f92e0 00000000`58b4acd8 00000000`26bdfc80 : rpcrt4!
CO_ConnectionThreadPoolCallback+0x13c
00000000`26bdf930 00007fff`68aec6f2 : 00007fff`65dd4c60 00000000`00000000 00000000`7ffe03b0 00007fff`00000000 : KERNELBASE!BasepTpIoCallback+0x59
00000000`26bdf980 00007fff`68b092c4 : 00000000`00000004 00007fff`00000000 0000000d`4fb34e30 00000000`26bdfc80 : ntdll!TppIopExecuteCallback+0x182
00000000`26bdfa10 00007fff`666313d2 : 00000000`00000000 00007fff`68b08a10 00000000`0c9f2090 00000000`00000000 : ntdll!TppWorkerThread+0x8b4
00000000`26bdfdf0 00007fff`68ae54f4 : 00007fff`666313b0 00000000`00000000 00000000`00000000 00000000`00000000 : kernel32!BaseThreadInitThunk+0x22
00000000`26bdfe20 00000000`00000000 : 00000000`00000000 00000000`00000000 00000000`00000000 00000000`00000000 : ntdll!RtlUserThreadStart+0x34
THREAD_SHA1_HASH_MOD_FUNC: 3943b14432cdd5f38f5055c79edc5148578ab120
THREAD_SHA1_HASH_MOD_FUNC_OFFSET: db5564c8ecd5b5a17f151de24887eddc14224475
THREAD_SHA1_HASH_MOD: a1f0419f278ce9387e9543b15b7f8ec11279ad60
FAULT_INSTR_CODE: 248c8b48
SYMBOL_STACK_INDEX: 1
SYMBOL_NAME: ax32serv!RaiseCriticalError+fe
FOLLOWUP_NAME: MachineOwner
MODULE_NAME: Ax32Serv
IMAGE_NAME: Ax32Serv.exe
DEBUG_FLR_IMAGE_TIMESTAMP: 557276dd
STACK_COMMAND: ~25s ; .ecxr ; kb
BUCKET_ID: ACCESS_VIOLATION_ax32serv!RaiseCriticalError+fe
FAILURE_EXCEPTION_CODE: c0000005
FAILURE_IMAGE_NAME: Ax32Serv.exe
BUCKET_ID_IMAGE_STR: Ax32Serv.exe
FAILURE_MODULE_NAME: Ax32Serv