##それはgcのknown problemだと思う。
#でもHEADをコンパイルするには0.9.2が入ってないとだめなので困りますな。早く0.9.3出さないと。
#やはりそうですか。0.9.3期待して待ってます。
#現在OSXで、HEADのext/tlsテストが止まる問題は原因がわかりました (axTLS自体のテストの問題)。FreeBSDの方はまだわからない。
#Hi Shiro!
#Are you around? =)
#yup
#Great
#So now that I've convinced myself that axTLS builds properly on the Mac, I'm getting a nice and predictable segfault in some axTLS code when running Gauche stuff via the C api
#It's happening downstream from Scm_TLSConnect, inside add_packet when calling MD5_Update
#Have you seen similar crashes at all?
#I'll note that TLS-using http transfers work 100% fine via the repl
#(looking at the code...)
#tls1.c, in function add_packet
#when I print the value of the ssl pointer, it's quoted as "0x18"... which is probably not good =)
#Mmmm gc hazard again?
#yeah perhaps
#seems to be supported by the idea that it works fine in the REPL
#why don't I disable GC and get back to you =)
#ScmTLS has finalizer attached, so you can also put a printf there to see if it is collected prematurely.
#yeah
#but why would that happen?
#anyway let me turn off GC first so that we know we're looking at the right idea
#hmm, xcode crashed. everything is crashing here =P
#yes, so the SSL pointer is 0
#... even when I call GC_Disable(). maybe gauche calls enable somewhere?
#No I don't think gauche turn on gc implicitly.
#right
#it's strange because as I go down the stack things are okay, but then I hit send_packet and suddenly the ssl pointer is 0
#So somebody's overwriting it maybe?
#but I imagine the TLS code is thread safe...
#plus it does work in the REPL
#any ideas of what else could be happening?
#Above, you said it crashes in Scm_TLSConnect. So the stack is Scm_TLSConnect -> ssl_client_new -> do_client_connect?
#sec, I'll provide a proper dump
#add_packet [inlined] at /Users/k/sources/gauche/ext/tls/axTLS/ssl/tls1.c:747
send_packet ()
send_client_hello [inlined] ()
do_client_connect ()
ssl_client_new ()
Scm_TLSConnect ()
rfc__tls_25tls_connect ()
run_loop ()
user_eval_inner ()
Scm_ApplyRec ()
#And at the time it reaches add_packet, ssl pointer gets 0?
#well, I don't think the stack trace is meaningful
#I just ran it with a slightly different thread count, and suddenly it's crashing with a segfault in some other place
#I am sure the code in axTLS is correct. The issue must be somewhere else
#ok.
#ext/tls code looks innocent... so some thread handling code in Gauche is doing silly thing...
#such as?
#no idea yet. but apparently somebody steps on to an unrelated memory location.
#hmm
#if there's a bug that overwrites specific offset of ScmTLS (somehow), then one probe is to add dummy fields in ScmTLS definition between SCM_HEADER and other fields and see if crash pattern changes (meaningfully)... of course if the bug overwrites random places it won't give a clue, but...
#hmm
#why does it work in the interpreter though?
#running it in a single thread only in my program also crashes it
#so it may not even be a thread issue. I'm truly perplexed.
#is axtls built with the same compiler gauche is built with, in the Makefiles?
#because I set CC and CXX before I run configure, so I hope those get propagated to axTLS?
#good point, let me check...
#well actually you can look at the make output?
#where can I check this?
#I mean, without rerunning make =)
#you don't need to. it does use the same CC, for it doesn't use axTLS's makefile
#so could be that axtls needs some special flag and gauche isn't giving it?
#who knows
#the config file is identical to the one generated with menuconfig
#axTLS original source does have some conditional stuff to add CFLAGS but I don't see anything special there
#it's got some curious LDFLAGS.. are those relevant to gauche?
#which is... (I don't see OSX specific stuff, do you use Linux one?)
#yeah, essentially the same flags
#except the "soname" flag, which needs to be "install_name" on the Mac
#are we linking axtls statically?
#is it remarkable at all that the function "send_packet" where ssl == 0 on entry (before anything happens) is also the first function in the trace that's inlined?
#well, gauche's makefile compiles individual axtls .c files into .o, then links them together with gauche-specific files to make rfc--tls.so.
#so soname won't matter in our case, I think
#maybe it's a compiler error?
#I'm using clang, not gcc
#If we want to make sure the compiler is somehow getting in our way... you can edit ext/Makefile.ext to remove -O2 and -fomit-frame-pointer, and in ext/tls run make clean; make, and see the difference. (But axTLS original makefile does use both flags, though)
#hmm. sounds dubious but I'll try
#yeah, no difference.
#maybe I'll write a tiny isolated test program that will do a single TLS request and link against my build..
#I don't see any clue now. More isolation needed, I agree.
#but... with the -O0 build there's no more inlining, and now it turns out that ssl isn't null after all
#the debugger was getting confused by the inlined function so it thought that ssl was 0... silly gdb
#and you still get crash?
#so it's crashing in MD5_Update
#ok so let's dump the memory overwriting hypothesis :)
#MD5_Update is in crypto.h
#maybe it's including the wrong header if say, openssl also has crypto/crypto.h? =)
#or maybe there are two functions with the same name for some reason..
#OpenSSL does have a function called MD5_Update =P
#There are.
#I remember I had a similar problem before...
#So maybe it's calling the wrong one and obviously crashing =)
#the project does link against libcrypto dynamically
#you can add #define MD5_Update MD5_Update_axtls in some header and see the difference maybe
#Actually Gauche also has MD5_Update in ext/digest. If rfc.md5 is loaded it is loaded, too.
#I'm assuming this is a problem?
#well let's see... if it was calling the wrong function, would this be visible in the symbol table of the rfc--tls.o file?
#wouldn't we see external references to MD5_Update ?
#well, i don't know. as far as the signature and MD5_CTX definition matches, it shouldn't be a problem even if it is calling another one.
#(because there aren't any references to MD5 in rfc--tls.o's symbol table)
#heh, so it's running the wrong code for sure
#rfc--tls.o doesn't call it directly (it is merely a compilation of rfc--tls.c, which is generated by tls.scm). you meant rfc--tls.so?
#yeah
#I'm looking through the asm dump that's crashing, and it makes references to a function called "md5_block_data_order"
#... which does not exist in Gauche or axTLS
#the symbol is present in openssl, though =)
#so how do I force it to call the right one? this is lunacy =)
#the easier path is to rename *our* MD5_* stuff (by #define magic)
#hmm... so should I add it to axTLS?
#could I specify this as part of the build command line?
#Let's see if you can edit ext/Makefile.ext to add -DMD5_Update=MD5_Update_axtls etc into CFLAGS...
#Just to see if that's the cause.
#hmph
#Well hold on, let's think about it more
#The issue is that I have an executable that links against both libcrypto and rfc--tls.so , and both contain MD5_Update. correct?
#I'll put a print message inside md5_update from axTLS and we'll see =)
#yeah, let's make sure what we think is actually happening.
#yes it's not calling the right one
#the address of the instruction in the debugger where the crash happens is not the address of MD5_Update when the symbol is printed
#... and my little print hello message didn't get printed, either.
#But what's weird is that sometimes it works and sometimes it crashes, right?
#well
#I haven't gotten it to work yet =)
#but changing thread stuff causes different crashes
#presumably due to stuff getting loaded in a different order
#DLL hell all over again =)
#ah ok. Is it difficult to try renaming axtls's MD5_* stuff (and maybe SHA1_* stuff as well) with #define magic?
#yes, because the author of axTLS thought it would be funny to mirror the OpenSSL interface directly as a base feature, not an addon
#so many other functions have names that coincide with other OpenSSL stuff... it's a total disaster.
#I wonder if a linker flag could fix the problem, or something
#ah so it's not only MD5_* functions but it's all over.
#yes.
#everything inside axTLS/crypto is potentially dangerous.
#I'll make sure it's not including the wrong "crypto.h" file
#There should be some flags, but the treatment of symbol crashes among dlopen'ed modules differ among OSes...
#since they have the same signature, the linker won't mind since it's a .o file
#I don't think this is a runtime issue
#I think at compile time (or link time) it calls MD5_Update from the wrong library
#maybe the "-undefined suppress" is not contributing here =)
#Hmm. If this is indeed the culprit, I may be able to preprocess axtls source to replace dangerous symbols (like I'm doing in ssltest.c)
#hmm
#that would be not bad, although the preprocessing would be heavy and annoying to maintain
#yeah... if one linker flag solves this, that would be much better.
#perhaps if object files from crypto were listed before the object files from ssl, things would work better
#... considering that it's a one-pass linker
#Could you try? That sounds fragile, but who knows...
#=)
#ok, rebuilding
#yeah not surprisingly that didn't work =)
#d'oh. I gotta leave for a couple of hours. I might write up some code to rename crypto things later.
#hmm perhaps renaming is the right thing to do... but sounds nasty
#yeah I gotta' go anyway
#night!
Gauche > Archives > 2012/02/26