Ir al contenido principal

Entradas

Mostrando entradas de 2018

Stop measuring time with time.clock() in Python3

If we read the time.clock() documentation ( https://docs.python.org/3/library/time.html#time.clock): Deprecated since version 3.3: The behaviour of this function depends on the platform: use perf_counter() or process_time() instead, depending on your requirements, to have a well defined behaviour. I use time.perf_counter, but feel free to use time.process_time if fits for your requirements. The main difference they have are: time.perf_counter() It does include time elapsed during sleep and is system-wide. time.process_time() It does not include time elapsed during sleep. It is process-wide by definition.

A simple python async spider (async programming with python 3.6 step 2)

As a second step learning async programming I developed a very simple spider with python 3.6 and asynchronous developing. In this case the spider request a bunch of urls. The server that is serving waits on each request. http://localhost:8000/1 The number '1' tells the number of seconds that the server should wait: http://localhost:8000/2 -> makes server wait for 2 seconds http://localhost:8000/5 -> makes server wait for 5 seconds I'm testing with a server that makes to wait my consumer, feel free to use random waits or whatever you prefer. The consumer accepts two queues, a queue for urls to retrieve, and a queue to store the results. At the moment the consumer don't store nothing at the urls queue, only retrieve the urls configured on the hardcoded urls list. More functionalities will be added in the future. In this example aiohttp==2.3.10 is used. (iospider) $ pip install "aiohttp==2.3.10" import asyncio from contextlib import

First approach to the python3.6 async programming, a simple consumer/producer (async programming with python 3.6 step 1)

As a first approach to the async programming I'm developing small scripts with python. The main thing I need to learn is how to develop an asynchronous consumer/producer using an async Queue and python3.6. First, a reproduction of the problem with synchronous programming, later the solution I found with async programming. Synchronous # First attempt, synchronous from queue import Queue from time import sleep , perf_counter def produce (q: Queue , n: int ): for i in range (n): print ( f'produce {i}' ) q.put(i) sleep( 0.1 ) def consume (q: Queue): for i in q.queue: print ( f'consume {i}' ) sleep( 0.1 ) start = perf_counter() q = Queue() produce(q , 10 ) consume(q) print (perf_counter() - start) # Output produce 0 produce 1 produce 2 ... # imagin that consume 7 consume 8 consume 9 2.005685806274414 It takes about 2 seconds Asynchronous # Second attempt, Asynchronous import asyncio from

Stop python debugger exactly on the failing loop iteration

This post is for ipdb users. Is this case familiar for you?: Write a break point on your code Go into a loop Start pressing 'c' and 'n' until you found the iteration that is failing. It can works, but you can only do this ... one or two times maybe?? If you need to this frequently, this method doesn't works and you need another tool, but .... what's the tool I need? You need launch_ipdb_on_exception ... ... from ipdb import launch_ipdb_on_exception with launch_ipdb_on_exception(): for i in range ( 9 ): print (i) if i== 4 : raise Exception('Debug time!') ... # Output 0 1 2 3 4 Exception('Debug time!',) > <ipython-input-1-4f44dca149ad>(7)<module>() 5 print(i) 6 if i==4: ----> 7 raise Exception('Debug time!') ipdb> i # Note ipdb console!! 4 ipdb> c # Continues the execution In [2]: i # Ipython console again Out[2]: 4 In [3

How to add an import to multiple python files | Añadir import a muchos ficheros python

If you want to add an import line to one or multiple python files, you can achieve this easily using the isort package. Install isort in your python environment: pip install -r isort   Now, execute the next isort command to add an import to multiple python files. Multiple options can be added to configure how isort works. For example, imagine you have some changes on your working copy that cause an error because you need an import. If you write this files into a "files.txt" file (one line per file) Example /tmp/files.txt file1.py controllers/file2.py Command line FILES=$(cat /tmp/files.txt) isort -a "from foo import bar" ${FILES} As I say before, you can mix with a lot of options, for example this is the command I use to handle line length, indentation, third parties, ... isort -ac -tc -m 3 -w 119 -i4 -up -y ${FILES} Hope you found this useful!!

PyConEs 2017 overall

In September 2017 Ebury went to PyCon Spain 2017 which took place in Cáceres (Extremadura), at the beautiful location of San Francisco’s Cultural Complex. Read on for insights on refactoring, unicode, serverless, testing factories, diversity in the work place and open source! To read more about the PyCon Spain 2017, visit this page!! : https://labs.ebury.rocks/2017/10/31/pycones-2017-overall/