Python on the art of simplicity

Stanza - A Python NLP Library for Many Human Languages

Sun, 12 Apr 2020 00:00:00 +0000

I tested out Stanza. English tokenizer and definately works. I ran quick test with Japanese lang and output was somewhat unexpected.

import stanza

# japanese "ja", for english model "en"

stanza.download("ja")
nlp = stanza.Pipeline("ja")
doc = nlp("皆さんおはようございます！　ご機嫌いかがですか？")

for i, sentence in enumerate(doc.sentences):
 print(f"===== Sentence {i+1} tokens =====")
 print(*[f"word: {word.text}\t upos: {word.upos} xpos: {word.xpos}" for word in sentence.words], sep="\n")

The output is:

Install GraalVM and run ptyhon with debugger

Sun, 14 Apr 2019 00:00:00 +0000

What is GraalVM?

GraalVM is a high-performance, embeddable, polyglot virtual machine for running applications written in JavaScript, Python, Ruby, R, JVM-based languages like Java, Scala, Kotlin, and LLVM-based languages such as C and C++.

Here is the Official doc link

Hmm… Okay, I have to see it.

Let’s install

Below is the way I installed GraalVM Community Edition on Ubuntu 18.04. For other platform, the official doc installation guide is here.

# update this number to latest version from here: https://github.com/oracle/graal/releases
version=1.0.0-rc15
wget https://github.com/oracle/graal/releases/download/vm-${version}/graalvm-ce-${version}-linux-amd64.tar.gz
tar -xvzf graalvm-ce-${version}-linux-amd64.tar.gz
# clean up
rm graalvm-ce-${version}-linux-amd64.tar.gz
# to wherever you want. 
mv graalvm-ce-1.0.0-rc15/ ~/bin/graalvm
# if you want to make it permanent, put this in your bashrc 
export PATH=$HOME/graalvm/bin:$PATH

Now that your graalvm/bin in your path, you’ll get the GraalVM versions of those runtimes.

Install Python on Ubuntu

Mon, 15 Oct 2018 00:00:00 +0000

Check the latest version here At the time of writing, 3.8.0 is the latest and 3.8.5 has release candidate.

Also make sure you have sqlite3, libbz2-dev and libffi-dev are installed

sudo apt-get install libsqlite3-dev libbz2-dev libffi-dev

version=3.8.5
wget https://www.python.org/ftp/python/${version}/Python-${version}.tgz
tar xzvf Python-${version}.tgz
cd Python-${version}
# Linux (or any Unix-like system), the default prefix and exec-prefix are /usr/local.
# thus you should be able to omit --prefix here
# --enable-optimizations option for significant speed boost (10-20%) but much
# slower build process
./configure --prefix /usr/local --enable-optimizations
make
sudo make install
# OR if you want to skip creating the python link then:
sudo make altinstall

in case you want to remove and re-install it again cause some software was missing before installation

Intercept Python Logging

Thu, 26 Jul 2018 00:00:00 +0000

The problem:

I need to ship specific log record and had formatter written in python. It is pretty complex transformation.

I thought of using Logstash but I then need to either convert this python logic or write a plugin to use already written python parser. Plus I need to install logstash… I wanted a simpler solution

How to solve it

Use custom python logging Handler and Filter!

import logging

messages = []
logger = logging.getLogger(__name__)
logger.setLevel(logging.DEBUG)


class ListenFilter(logging.Filter):

 def filter(self, record):
 """Determine which log records to output.
 Returns 0 for no, nonzero for yes.
 """
 if record.getMessage().startswith('dont: '):
 return False
 return True


class RequestsHandler(logging.Handler):
 def emit(self, record):
 """Send the log records (created by loggers) to
 the appropriate destination.
 """
 messages.append(record.getMessage())


handler = RequestsHandler()
logger.addHandler(handler)

filter_ = ListenFilter()
logger.addFilter(filter_)

# log I want
logger.info("logme: Howdy!")


# log i want to skip
logger.info("dont: I'm doing great!")

# prints ['logme: Howdy!']
print(messages)

Cheers!

Python Project Install - develop vs install & setuptools vs pip

Mon, 05 Mar 2018 00:00:00 +0000

The problem:

I don’t understand the difference between setup.py develop and setup.py install
I don’t understand the difference between setup.py develop and pip install -e [dir]
I don’t see the changes to my code when I import my code

The difference between `setup.py develop` and `setup.py install`

In short, you want to run setup.py develop when you are editing code because when you run setup.py install, it will copy your code into site-packages thus if you want to test your latest code you will need to install (copy) again. On the other hand, with develop, it creates a link to your source code so that when you import your code, it is your latest code.

No Space Left Error when pip install

Tue, 26 Dec 2017 00:00:00 +0000

The Problem:

Got

OSError: [Errno 28] No space left on device

When your home directory where your virtual environment located has more than enough space…

Why?:

The culprit is likely that your /tmp directory do not have enough space for some reason. During the pip installation, pip will use temporarily directory to perform what is necessary to perform installation (e.g. download source etc). Thus if you do not have enough space in /tmp that package installation requires then you will get disk space error.

Python on the art of simplicity

Stanza - A Python NLP Library for Many Human Languages

Install GraalVM and run ptyhon with debugger

What is GraalVM?

Let’s install

Install Python on Ubuntu

Intercept Python Logging

The problem:

How to solve it

Python Project Install - develop vs install & setuptools vs pip

The problem:

The difference between setup.py develop and setup.py install

No Space Left Error when pip install

The Problem:

Why?:

The difference between `setup.py develop` and `setup.py install`