Tag Archives: unix

Inspecting thread dumps of hung Python processes and test runs

Posted on by
3

Sometimes, moderately complex Python applications with several threads tend to hang on exit. The application refuses to quit and just idles there waiting for something. Often this is because if any of the Python threads are alive when the process tries to exit it will wait any alive thread to terminate, unless Thread.daemon is set to true.

In the past, it use to be little painful to figure out which thread and function causes the application to hang, but no longer! Since Python 3.3 CPython interpreter comes with a faulthandler module. faulthandler is a mechanism to tell the Python interpreter to dump the stack trace of every thread upon receiving an external UNIX signal.

Here is an example how to figure out why the unit test run, executed with pytest, does not exit cleanly. All tests finish, but the test suite refuses to quit.

First we run the tests and set a special environment variable PYTHONFAULTHANDLER telling CPython interpreter to activate the fault handler. This environment variable works regardless how your Python application is started (you run python command, you run a script directly, etc.)

PYTHONFAULTHANDLER=true py.test

And then the test suite has finished, printing out the last dot… but nothing happens despite our ferocious sipping of coffee.

dotdotdotmoredotsthenthenthedotsstopappearing 
..

How to proceed:

Press CTRL-Z to suspend the current active process in UNIX shell.

Use the following command to send SIGABRT signal to the suspended process.

kill -SIGABRT %1

Voilá – you get the traceback. In this case, it instantly tells SQLAlchemy is waiting for something and most likely the database has deadlocked due to open conflicting transactions.

Fatal Python error: Aborted

Thread 0x0000000103538000 (most recent call first):
  File "/opt/local/Library/Fra%                                                                                                                                                                     meworks/Python.framework/Versions/3.4/lib/python3.4/socketserver.py", line 154 in _eintr_retry
  File "/opt/local/Library/Frameworks/Python.framework/Versions/3.4/lib/python3.4/socketserver.py", line 236 in serve_forever
  File "/Users/mikko/code/trees/pyramid_web20/pyramid_web20/tests/functional.py", line 40 in run
  File "/opt/local/Library/Frameworks/Python.framework/Versions/3.4/lib/python3.4/threading.py", line 921 in _bootstrap_inner
  File "/opt/local/Library/Frameworks/Python.framework/Versions/3.4/lib/python3.4/threading.py", line 889 in _bootstrap

Current thread 0x00007fff75128310 (most recent call first):
  File "/Users/mikko/code/trees/venv/lib/python3.4/site-packages/SQLAlchemy-1.0.0b5-py3.4-macosx-10.9-x86_64.egg/sqlalchemy/engine/default.py", line 442 in do_execute
...
  File "/Users/mikko/code/trees/venv/lib/python3.4/site-packages/SQLAlchemy-1.0.0b5-py3.4-macosx-10.9-x86_64.egg/sqlalchemy/sql/schema.py", line 3638 in drop_all
  File "/Users/mikko/code/trees/pyramid_web20/pyramid_web20/tests/conftest.py", line 124 in teardown
...
  File "/Users/mikko/code/trees/venv/lib/python3.4/site-packages/_pytest/config.py", line 41 in main
  File "/Users/mikko/code/trees/venv/bin/py.test", line 9 in <module>

\ Subscribe to RSS feed Follow me on Twitter Follow me on Facebook Follow me Google+

psutil – advanced OS process management utilities for Python

Posted on by
Reply

psutil is a Python library and API for UNIX system administration utilities. It provides Pythonic API e.g. for top, lsof, netstat and kill like tasks.

The benefits of using this library include

  • Because psutil API resembles UNIX process management command line utilities the developers find themselves home
  • Pythonic, easy to use, API
  • Cross-platform
  • Good documentation

psutil is maintained by Giampaolo Rodola and Jay Loden.

Below is a short example what you can do with psutil. It checks if the current operating system is running Apache (web server) process and this process is listening to particular TCP/IP ports. The use case could be e.g. per-requirements check that the software can be deployed against a particular system.

import psutil

def is_apache_running_in_ports(process_name="apache", ports=(80, 443)):
    """
    Check if a local Apache instance is running and listening to certain ports.

    :param ports: List of ports Apache should be listening to (all ports must be included)
    """

    ports_to_go = list(ports)

    # Iterate over all system processes (ps)
    for proc in psutil.process_iter():
        if proc.name != process_name:
            # Not target process
            continue

        # Iterate over all ports this process is listening to 
        for con in proc.get_connections():
            # Tuple ip, port
            port = con.local_address[1]
            if port in ports_to_go:
                ports_to_go.remove(port)

    # Did we find all ports we wanted to
    return len(ports_to_go) == 0
Note: When installing psutil be sure that you follow the Python best practices: sudo-free virtualenv installation.

More examples at psutil homepage.

 

\ Subscribe to RSS feed Follow me on Twitter Follow me on Facebook Follow me Google+

Useful duckduckgo searches for developers

Posted on by
5

DuckDuckGo.com is a web search engine. The superior value of DuckDuckGo comes from its ability of doing specialized searches. For non-specialized searches it uses Bing as the generic web search backend. In this blog post I describe how you can optimize your web and software development workflow with DuckDuckGo and its specialized searches.

Bing is almost on par with Google for English search results. However, it still leaves much improvement for non-English search results like ones with my native language Finnish. Thus, one cannot use DuckDuckGo for your everyday searches, like for shopping in Finland. You still need to go to Google for those. To work around this issue I have dual search engine configuration in my Firefox.

  • If I want to search from Google I type the search on the address bar (left side).
  • If I want a specialized search I type into the DuckDuckGo search box. (right side) You can install it for your browser from a button on the bottom right corner of duckduckgo.com.
  • Please note that this setup does not work with Safari which cannot do address bar searches.  Chrome also lacks the search box widget out of the box. There is something called Search Box for Chrome, but I have not tried to configure it for DuckDuckGo on Chrome (I use Chrome only as the web development browser).

1. Specialized searches

DuckDuckGo provides something called !bang syntax. You start your search query with a bang (!) and a keyword. In this case DuckDuckGo performs a special search, not a generic web search. Usually this means DuckDuckGo uses a site specific search engine skipping the web search altogether. Here are example queries which you will find very useful.

!mdn iframe – Search term iframe on Mozilla Developer Network. Useful for all HTML, CSS and Javascript related searches. MDN is the most hiqh quality information source for all web development related matters. Using this search skips low quality sites like w3schools.com. Also works: !javascript and !css.

Example what happens when you search !mdn iframe in duckduckgo.com search box

!python socket – Search “socket” on docs.python.org for Python documentation. (no one remembers Python APIs)

!pypi vvv – Search a package named vvv on pypi.python.org (a validation and linting integration for your software development projects)

!jq offset – Search offset function in jQuery documentation

!gh sevabot – Search Github for projects called “sevabot” (It’s a very handy Skype bot with HTTP interface)

!so video canvas – Search stackoverflow.com questions related to “video” and “canvas” (how to draw HTML5 video frames on <canvas>)

!php open – Try to figure out how to open a file in PHP (there are always open positions in non-PHP related software development jobs if you feel your life is misery)

!man zsh – UNIX manual page for zsh command. (check also our tuned ZSH configuration)

!ubuntuforums zsh – Search ubuntuforums.com for threads related to ZSH

!yt wesibussi – Search YouTube videos called “wesibussi” (I was young and it was fun)

As you can see, you can also copy-paste the DuckDuckGo.com search link in chat (IRC) easily for less experienced peers to educate them how to do power searches.

For the full bang queries list see the official duckduckgo.com source. You will find useful pointers for other programming languages (Perl, Ruby, Java, C#, etc).

 

 

\ Subscribe to RSS feed Follow me on Twitter Follow me on Facebook Follow me Google+

Five ZSH tricks to optimize your shell workflow

Posted on by
5

Below on the video I’ll demonstrate five handy tricks you can do with zsh shell. zsh is a powerful command line shell for UNIX operating systems (Linux, Ubuntu, OSX, etc.) Newcomer shell users may find these tips helpful.

You want to use zsh because it make you type less, find command faster and ultimately make you more productive when working with UNIX and servers.

The tips are based on our tuned ztanesh shell scripts which you may wan to install too 🙂

1) Navigate around on the file system using zsh “menu select” tab completion. Writing a path to the command line equals to cd‘ing that directory in zsh. What we do here is that we open the tab completion in the current folder using the dot slash notation and then find our way to the target folder. The menu select constructs the path for us. Instructions how to set up zsh’s menu select for this.

2) Back to the previous folder where you were: cd –. Handy when you jump around in the directory tree when working. You can type only cd to go to the home folder. .. (even without cd) takes you one level higher, … two lever higher and so on.

3) Move a file using the menu select completion as shown in the first trick. You can open menu completion for any command or target.

4) zsh has superior tab completion compared to other shells: here I autocomplete an SSH host and use the menu selector to choose it from the list. zsh can also complete scp copy remote filenames. Other example is kill command where the tab completion allows you to choose the target process in a menu.

5) CTRL+R for the history search. Type CTRL + R and few letters of any of your previous commands and CTRL+R again to find it up (incremental search with history-incremental-pattern-search-backward). No need to navigate through the long backlog using up arrow. I don’t know how I managed to live 15 years without getting this to my muscle memory.

Also on the video you can see how the terminal tab colorization by the current (ssh‘ed) server works.

For more useful keybindings on OSX / iTerm 2, see how to bind ALT + arrow to move between words and fn + arrow to move to the beginning and the end of the line.

More tips welcome. Especially I am interested in

  • Fuzzy / partial string match for the history earch (Sublime Text 2 CTRL+T style go to anywhere)
  • Menu select for the history search

 

\ Subscribe to RSS feed Follow me on Twitter Follow me on Facebook Follow me Google+