I have a huge database (of 500GB or so) an was able to put it in pandas. The databasse contains something like 39705210 observations. As you can imagine, python has hard times even opening it. Now, I am trying to use Dask in order to export it to cdv into 20 partitions like this: However when I am trying to
Tag: python
How to pass URL as a path parameter to a FastAPI route?
I have created a simple API using FastAPI, and I am trying to pass a URL to a FastAPI route as an arbitrary path parameter. When I test it, it doesn’t work and throws an error. I am testing it this way: Answer Option 1 You can simply use the path convertor to capture arbitrary paths. As per Starlette do…
How can i make that when a poll reaches por example 5 votes send the message to another channel
I’m trying to make a poll, but all the ones I see have a limit of time created with asyncio, but I want to make that when the a poll reaches 5 reaction of the emoji ✅ instead of having a limit of time, send the message to another channel, I already have the poll itself done How can I
How to perform operations on very big torch tensors without splitting them
My Task: I’m trying to calculate the pair-wise distance between every two samples in two big tensors (for k-Nearest-Neighbours), That is – given tensor test with shape (b1,c,h,w) and tensor train with shape (b2,c,h,w), I need || test[i]-train[j] || for every i,j. (where both test[i] and train[j] h…
Due to huge images. Can we generalize flow_from_directory method for regression problem (CNN): input image and output is x,y (float)
Data is in a CSV file which includes image path and target (x and y). where x and y belong to [-1 to 1] after scale (in keras becasue there are so many image . I can not load all in X_train like normal). Thank you so much for help! data in csv file Answer I will write here as
DNN OpenCV Python using RSTP always crash after few minutes
Description: I want to create a people counter using DNN. The model I’m using is MobileNetSSD. The camera I use is IPCam from Hikvision. Python communicates with IPCam using the RSTP protocol. The program that I made is good and there are no bugs, when running the sample video the program does its job w…
How to scrape a page that is dynamicaly locaded?
So here’s my problem. I wrote a program that is perfectly able to get all of the information I want on the first page that I load. But when I click on the nextPage button it runs a script that loads the next bunch of products without actually moving to another page. So when I run the next loop all
Connect two comboboxes via a dictionary (with the purpose to display male or female names in the combobox)
I have a combobox where I select the gender Male or Female. Then I have another combobox where now all the names of people are displayed (without distinction of gender). I would like to select Male from the first combobox and automatically display (without button) the Male names in the second combobox. The sa…
Groupby column and create lists for other columns, preserving order
I have a PySpark dataframe which looks like this: I want to group by or partition by ID column and then the lists for col1 and col2 should be created based on the order of timestamp. My approach: But this is not returning list of col1 and col2. Answer I don’t think the order can be reliably preserved us…
How to use GTK4 without using the `Application` class?
Note how this application implements a mainloop by the activate signal and .run() method call: Correct me: I think this activate signal is emitted as an event to synchronise the creation of GTK windows, widgets, etc, so that the GUI part does not race to happen before the application is ready. What else is th…