Ir al contenido principal

Entradas

Polynomial regression using python

We are going to learn how to create a polynomial regression and make a prediction over a future value using python. The data set have been fetched from INE (national statistics institute) , that data is the EPA ( active population survey ), that tell us the national total (Spain), both genders. 16 and over are unemployed ( in thousands ). Example data: label serie rate 0 2002T1 0 2152.8 1 2002T2 1 2103.3 2 2002T3 2 2196.0 3 2002T4 3 2232.4 4 2003T1 4 2328.5 Data CSV can be downloaded here: https://drive.google.com/file/d/1fwvAZe7lah5DX8-DDEpmfeUDYQhKcfzG/view?usp=sharing Lets see how looks that data: Fine, as we can see the data describe a curve, so its for that because we want to use a polynomial regression. To try to approximate that curve we will use a grade 2 polynomial or
Publicar un comentario
Leer más
Entradas recientes

Use django ORM standalone within your nameko micro-services

Learning about micro services with python, I found a great tool named nameko . https://www.nameko.io/ Nameko is a Python framework to build microservices that doesn't care in concrete technologies you will use within your project. To allow that microservices to work with a database, you can install into your project a wide variety of third parties, like SQLAlchemy (just like any other). To have an easy way to communicate with the database and keep track of the changes made to the models, I chose Django: I'm just learning about microservices and I want to keep focused on that. Easy to use, Django is a reliable web framework, have a powerful and well known ORM. Also using Django we will have many of the different functionalities that this framework provide. To make all this magic to work together, I developed a python package that allow you to use Django as a Nameko injected dependency: https://pypi.org/project/django-nameko-standalone/ You can found the source
Publicar un comentario
Leer más

Stop measuring time with time.clock() in Python3

If we read the time.clock() documentation ( https://docs.python.org/3/library/time.html#time.clock): Deprecated since version 3.3: The behaviour of this function depends on the platform: use perf_counter() or process_time() instead, depending on your requirements, to have a well defined behaviour. I use time.perf_counter, but feel free to use time.process_time if fits for your requirements. The main difference they have are: time.perf_counter() It does include time elapsed during sleep and is system-wide. time.process_time() It does not include time elapsed during sleep. It is process-wide by definition.
Publicar un comentario
Leer más

A simple python async spider (async programming with python 3.6 step 2)

As a second step learning async programming I developed a very simple spider with python 3.6 and asynchronous developing. In this case the spider request a bunch of urls. The server that is serving waits on each request. http://localhost:8000/1 The number '1' tells the number of seconds that the server should wait: http://localhost:8000/2 -> makes server wait for 2 seconds http://localhost:8000/5 -> makes server wait for 5 seconds I'm testing with a server that makes to wait my consumer, feel free to use random waits or whatever you prefer. The consumer accepts two queues, a queue for urls to retrieve, and a queue to store the results. At the moment the consumer don't store nothing at the urls queue, only retrieve the urls configured on the hardcoded urls list. More functionalities will be added in the future. In this example aiohttp==2.3.10 is used. (iospider) $ pip install "aiohttp==2.3.10" import asyncio from contextlib import
Publicar un comentario
Leer más

First approach to the python3.6 async programming, a simple consumer/producer (async programming with python 3.6 step 1)

As a first approach to the async programming I'm developing small scripts with python. The main thing I need to learn is how to develop an asynchronous consumer/producer using an async Queue and python3.6. First, a reproduction of the problem with synchronous programming, later the solution I found with async programming. Synchronous # First attempt, synchronous from queue import Queue from time import sleep , perf_counter def produce (q: Queue , n: int ): for i in range (n): print ( f'produce {i}' ) q.put(i) sleep( 0.1 ) def consume (q: Queue): for i in q.queue: print ( f'consume {i}' ) sleep( 0.1 ) start = perf_counter() q = Queue() produce(q , 10 ) consume(q) print (perf_counter() - start) # Output produce 0 produce 1 produce 2 ... # imagin that consume 7 consume 8 consume 9 2.005685806274414 It takes about 2 seconds Asynchronous # Second attempt, Asynchronous import asyncio from
Publicar un comentario
Leer más

Stop python debugger exactly on the failing loop iteration

This post is for ipdb users. Is this case familiar for you?: Write a break point on your code Go into a loop Start pressing 'c' and 'n' until you found the iteration that is failing. It can works, but you can only do this ... one or two times maybe?? If you need to this frequently, this method doesn't works and you need another tool, but .... what's the tool I need? You need launch_ipdb_on_exception ... ... from ipdb import launch_ipdb_on_exception with launch_ipdb_on_exception(): for i in range ( 9 ): print (i) if i== 4 : raise Exception('Debug time!') ... # Output 0 1 2 3 4 Exception('Debug time!',) > <ipython-input-1-4f44dca149ad>(7)<module>() 5 print(i) 6 if i==4: ----> 7 raise Exception('Debug time!') ipdb> i # Note ipdb console!! 4 ipdb> c # Continues the execution In [2]: i # Ipython console again Out[2]: 4 In [3
Publicar un comentario
Leer más

How to add an import to multiple python files | Añadir import a muchos ficheros python

If you want to add an import line to one or multiple python files, you can achieve this easily using the isort package. Install isort in your python environment: pip install -r isort   Now, execute the next isort command to add an import to multiple python files. Multiple options can be added to configure how isort works. For example, imagine you have some changes on your working copy that cause an error because you need an import. If you write this files into a "files.txt" file (one line per file) Example /tmp/files.txt file1.py controllers/file2.py Command line FILES=$(cat /tmp/files.txt) isort -a "from foo import bar" ${FILES} As I say before, you can mix with a lot of options, for example this is the command I use to handle line length, indentation, third parties, ... isort -ac -tc -m 3 -w 119 -i4 -up -y ${FILES} Hope you found this useful!!
Publicar un comentario
Leer más