I have a Python 2.7 / Flask app running on Heroku that scrapes data and I now want to store that information in a database. I’ve tried to follow tutorials like this one and apply this to my case but I can’t get it to work. I have created & promoted my postgres database successfully on heroku. …
Tag: python
How to iterate over consecutive chunks of Pandas dataframe efficiently
I have a large dataframe (several million rows). I want to be able to do a groupby operation on it, but just grouping by arbitrary consecutive (preferably equal-sized) subsets of rows, rather than using any particular property of the individual rows to decide which group they go to. The use case: I want to ap…
Disabling sorting mechanism in pprint output
I have big dictionary which I`m printing for viewing with prettyprint, but how I can keep formatting but kill sorting mechanism in pprint? Answer Python 3.8 or newer: Use sort_dicts=False: Python 3.7 or older: You can monkey patch the pprint module. Since the 2nd output is essentiallly randomly sorted, your o…
How to make a variable inside a try/except block public?
How can I make a variable inside the try/except block public? This code returns an error How can I make the variable text available outside of the try/except block? Answer try statements do not create a new scope, but text won’t be set if the call to url lib.request.urlopen raises the exception. You pro…
make os.listdir() list complete paths
Consider the following piece of code: The objective is to sort the listed files based on the creation time. However since the the os.listdir gives only the filename and not the absolute path the key ie, the os.path.getctime throws an exception saying OSError: [Errno 2] No such file or directory: ‘very_i…
Using Python Pandas to bin data in one df according to bins defined in a second df
I am attempting to bin data in one dataframe according to bins defined in a second dataframe. I am thinking that some combination of pd.bin and pd.merge might get me there? This is basically the form each dataframe is currently in: df: And this is the table with the bins, df2: I would like to match the bin, a…
The Next Palindrome number
I am beginner in programming, So can you please tell me what’s wrong with my code? I want to print next palindrome number if the number entered by the user (n) is not palindrome Answer There are two problems with your code. 1) Your “for i in range” loop calculates the reverse of the temp var…
multiple execute on a single connection.cursor in django. Is it safe?
I am opening a cursor with connection.cursor executing a bunch of deletes then closing the cursor. It works, but I am not sure if it has any side effect. Would appreciate any feedback. Answer Since you are not executing these SQL statements in the transaction, you may encounter confusing states (for example, …
Plotting CDF of a pandas series in python
Is there a way to do this? I cannot seem an easy way to interface pandas series with plotting a CDF. Answer In case you are also interested in the values, not just the plot. This will always work (discrete and continuous distributions) Alternative example with a sample drawn from a continuous distribution or …
Python classes self.variables
I have started learning python classes some time ago, and there is something that I do not understand when it comes to usage of self.variables inside of a class. I googled, but couldn’t find the answer. I am not a programmer, just a python hobbyist. Here is an example of a simple class, with two ways of…