how do i change the “keyword” in parameter (‘pegasus’) to redirect to a separate txt file. so later I just write whatever items I want to scrape in the file txt. Example of Pegasus, Phoenix, Lucid then the keyword parameter is directed to a different txt, inside the txt it contains a list of those objects and once it has finished scraping the pegasus it will scrape next item which is Phoenix, and so on
import requests import csv key = input('enter name file u want :') write = csv.writer(open('output/{}.csv'.format(key), 'w', newline='')) header = ['Name', 'Price', 'Seller', 'Order'] write.writerow(header) url = 'https://api-gateway.itemku.com/v1/product' count = 0 for page in range(1, 2): parameter = { 'is_include_game': 1, 'is_include_item_type': 1, 'is_include_item_info_group': 0, 'is_include_order_record': 1, 'is_from_web': 1, 'is_include_upselling_product': 1, 'game_id': 29, 'per_page': 18, 'page': page, 'sort': 'cheap', 'is_auto_delivery_first': 1, 'is_with_promotion': 1, 'keyword': 'pegasus' } r = requests.get(url, params=parameter).json() products = r['data']['data'] for p in products: name = p['name'] price = p['price'] seller = p['seller'] order = p['order_record'] count += 1 print('No :', count, 'name:', name, 'price:', price, 'seller:', seller, 'order:', order) write = csv.writer( open('output/{}.csv'.format(key), 'a', newline='')) data = [name, price, seller, order] write.writerow(data)
Advertisement
Answer
how do i change the “keyword” in parameter (‘pegasus’)
If you mean changing “value” of key “keyword” inside parameter, then simply replace the value:
parameter['keyword'] = "Pheonix"
To read data from a .txt file and loop through it, just read the file and add the values to list, then loop through it
key = [line.strip() for line in open('../relative/path','r') .... for keyword in key: .... parameter = { 'is_include_game': 1, 'is_include_item_type': 1, 'is_include_item_info_group': 0, 'is_include_order_record': 1, 'is_from_web': 1, 'is_include_upselling_product': 1, 'game_id': 29, 'per_page': 18, 'page': page, 'sort': 'cheap', 'is_auto_delivery_first': 1, 'is_with_promotion': 1, 'keyword': keyword } # or you could set the value after creating parameter dictionary # parameter['key'] = keyword
Reading a file this way will not close the file after it finished reading, But it shouldn’t be a problem unless you use this method to open a lot of files. garbage collector should do the job closing the file for us.