ruby - Hanging on rake db:migrate ts:conf -


i have jenkins running deploy script on of our app machines. lately, half builds don't finish , keep hanging while trying run same thing. last of output looks this:

 ** [app@app1 :: stdout] generating configuration /var/www/app/releases/20130509192657/config/production.sphinx.conf  ** [app@app2 :: stdout] generating configuration /var/www/app/releases/20130509192657/config/production.sphinx.conf  ** [app@app3 :: stdout] generating configuration /var/www/app/releases/20130509192657/config/production.sphinx.conf  ** [app@app4 :: stdout] generating configuration /var/www/app/releases/20130509192657/config/production.sphinx.conf  ** [app@app6 :: stdout] generating configuration /var/www/app/releases/20130509192657/config/production.sphinx.conf  ** [app@app7 :: stdout] generating configuration /var/www/app/releases/20130509192657/config/production.sphinx.conf 

app5 machine seems have problem, , occurs when tries run:

/usr/local/bin/ruby /usr/local/bin/bundle exec rake db:migrate ts:conf 

production running ruby 1.9.3p194, , due legacy reasons we're still running thinkingsphinx v. 0.9.8. we're running rails 3.2.13 , thinkingsphinx 2.0.7.

running strace on hanging process shows me this:

... 29802 select(4, [3], null, null, null <unfinished ...> 29790 restart_syscall(<... resuming interrupted call ...>) = -1 etimedout (connection timed out) 29790 futex(0x64a88e8, futex_wake_private, 1) = 0 29790 write(4, "!", 1 <unfinished ...> 29802 <... select resumed> )            = 1 (in [3]) 29790 <... write resumed> )             = 1 29802 read(3,  <unfinished ...> 29790 futex(0x1d47f64, futex_wait_private, 3, null <unfinished ...> 29802 <... read resumed> "!", 1024)     = 1 29802 select(4, [3], null, null, {0, 100000}) = 0 (timeout) 29802 select(4, [3], null, null, {0, 100000}) = 0 (timeout) 29802 select(4, [3], null, null, {0, 100000}) = 0 (timeout) 29802 select(4, [3], null, null, {0, 100000}) = 0 (timeout) 29802 select(4, [3], null, null, {0, 100000}) = 0 (timeout) 29802 select(4, [3], null, null, {0, 100000}) = 0 (timeout) 29802 select(4, [3], null, null, {0, 100000}) = 0 (timeout) 29802 select(4, [3], null, null, {0, 100000}) = 0 (timeout) 29802 select(4, [3], null, null, {0, 100000}) = 0 (timeout) 29802 select(4, [3], null, null, {0, 100000}) = 0 (timeout) 29802 select(4, [3], null, null, {0, 100000}) = 0 (timeout) 29802 select(4, [3], null, null, {0, 100000}) = 0 (timeout) 29802 select(4, [3], null, null, {0, 100000}) = 0 (timeout) 29802 select(4, [3], null, null, {0, 100000}) = 0 (timeout) 29802 select(4, [3], null, null, {0, 100000}) = 0 (timeout) 29802 select(4, [3], null, null, {0, 100000}) = 0 (timeout) 29802 select(4, [3], null, null, {0, 100000}) = 0 (timeout) 29802 select(4, [3], null, null, {0, 100000}) = 0 (timeout) 29802 select(4, [3], null, null, {0, 100000}) = 0 (timeout) 29802 select(4, [3], null, null, {0, 100000}) = 0 (timeout) ... 

has ever seen before? not having of background in sysops, there specific approach should taking in trying tackle problem?

if db:migrate locking, may have active, or hung - perhaps zombie - process locking database table (or other resource) being referred in migration. have experienced data fix-up script ran engineer had crashed (over week before deployment attempting), not exited - holding open transaction prevented alteration table. fix terminate stuck process, , migrate worked normal.

without knowing system architecture, it's difficult precise conflicting resource be. rdbms toolkit may allow take @ db hosted on server , see open connections to.


Comments

Popular posts from this blog

linux - xterm copying to CLIPBOARD using copy-selection causes automatic updating of CLIPBOARD upon mouse selection -

c++ - qgraphicsview horizontal scrolling always has a vertical delta -