Skip to content
Advertisement

How to collect and move data from a link to pandas Dataframe using request

I have a link as below:

https://beta.nseindia.com/api/snapshot-derivatives-equity?index=futures

I want to collect and move table data from a link to pandas Dataframe using request

Advertisement

Answer

Use requests.get first and then json.json_normalize:

import pandas as pd
import json
import requests
from pandas.io.json import json_normalize  

url = "https://beta.nseindia.com/api/snapshot-derivatives-equity?index=futures"

result = requests.get(url).json()
#print (result)


df = json_normalize(result['volume'], 'data','timestamp')
print (df)

    underlying                        identifier instrumentType  
0          DLF         FUTSTKDLF29-08-2019XX0.00         FUTSTK   
1      YESBANK     FUTSTKYESBANK29-08-2019XX0.00         FUTSTK   
2    LICHSGFIN   FUTSTKLICHSGFIN29-08-2019XX0.00         FUTSTK   
3   IBULHSGFIN  FUTSTKIBULHSGFIN29-08-2019XX0.00         FUTSTK   
4   TATAMOTORS  FUTSTKTATAMOTORS29-08-2019XX0.00         FUTSTK   
5         SBIN        FUTSTKSBIN29-08-2019XX0.00         FUTSTK   
6    ICICIBANK   FUTSTKICICIBANK29-08-2019XX0.00         FUTSTK   
7      YESBANK     FUTSTKYESBANK26-09-2019XX0.00         FUTSTK   
8   JINDALSTEL  FUTSTKJINDALSTEL29-08-2019XX0.00         FUTSTK   
9          DLF         FUTSTKDLF26-09-2019XX0.00         FUTSTK   
10        VEDL        FUTSTKVEDL29-08-2019XX0.00         FUTSTK   
11    ASHOKLEY    FUTSTKASHOKLEY29-08-2019XX0.00         FUTSTK   
12  BANKBARODA  FUTSTKBANKBARODA29-08-2019XX0.00         FUTSTK   
13        NMDC        FUTSTKNMDC29-08-2019XX0.00         FUTSTK   
14        SAIL        FUTSTKSAIL29-08-2019XX0.00         FUTSTK   
15         PNB         FUTSTKPNB29-08-2019XX0.00         FUTSTK   
16         NCC         FUTSTKNCC29-08-2019XX0.00         FUTSTK   
17        IDEA        FUTSTKIDEA29-08-2019XX0.00         FUTSTK   
18   TATAPOWER   FUTSTKTATAPOWER29-08-2019XX0.00         FUTSTK   
19    GMRINFRA    FUTSTKGMRINFRA29-08-2019XX0.00         FUTSTK   
20       NIFTY       FUTIDXNIFTY29-08-2019XX0.00         FUTIDX   
21       NIFTY       FUTIDXNIFTY26-09-2019XX0.00         FUTIDX   
22       NIFTY       FUTIDXNIFTY31-10-2019XX0.00         FUTIDX   
23   BANKNIFTY   FUTIDXBANKNIFTY29-08-2019XX0.00         FUTIDX   
24   BANKNIFTY   FUTIDXBANKNIFTY26-09-2019XX0.00         FUTIDX   
25   BANKNIFTY   FUTIDXBANKNIFTY31-10-2019XX0.00         FUTIDX   
26     NIFTYIT     FUTIDXNIFTYIT29-08-2019XX0.00         FUTIDX   
27     NIFTYIT     FUTIDXNIFTYIT31-10-2019XX0.00         FUTIDX   
28     NIFTYIT     FUTIDXNIFTYIT26-09-2019XX0.00         FUTIDX   

       instrument   expiryDate optionType  strikePrice  lastPrice     pChange  
0   Stock Futures  29-Aug-2019          -            0     146.90  -14.617844   
1   Stock Futures  29-Aug-2019          -            0      61.90   -5.351682   
2   Stock Futures  29-Aug-2019          -            0     429.00   -6.931337   
3   Stock Futures  29-Aug-2019          -            0     441.25   -7.832898   
4   Stock Futures  29-Aug-2019          -            0     110.70   -1.512456   
5   Stock Futures  29-Aug-2019          -            0     276.25   -0.575850   
6   Stock Futures  29-Aug-2019          -            0     407.35   -1.356096   
7   Stock Futures  26-Sep-2019          -            0      61.95   -5.275229   
8   Stock Futures  29-Aug-2019          -            0      98.40   -0.656234   
9   Stock Futures  26-Sep-2019          -            0     147.20  -14.765489   
10  Stock Futures  29-Aug-2019          -            0     135.05   -3.397711   
11  Stock Futures  29-Aug-2019          -            0      60.45   -2.578566   
12  Stock Futures  29-Aug-2019          -            0      92.80   -2.161307   
13  Stock Futures  29-Aug-2019          -            0      83.20   -3.981535   
14  Stock Futures  29-Aug-2019          -            0      31.35    0.000000   
15  Stock Futures  29-Aug-2019          -            0      62.60   -2.263856   
16  Stock Futures  29-Aug-2019          -            0      52.40   -4.640582   
17  Stock Futures  29-Aug-2019          -            0       5.55    0.000000   
18  Stock Futures  29-Aug-2019          -            0      53.25    4.105572   
19  Stock Futures  29-Aug-2019          -            0      14.70    0.000000   
20  Index Futures  29-Aug-2019          -            0   10852.60   -0.732210   
21  Index Futures  26-Sep-2019          -            0   10889.05   -0.779527   
22  Index Futures  31-Oct-2019          -            0   10935.80   -0.716315   
23  Index Futures  29-Aug-2019          -            0   27506.55   -0.840850   
24  Index Futures  26-Sep-2019          -            0   27617.45   -0.836611   
25  Index Futures  31-Oct-2019          -            0   27715.00   -0.733707   
26  Index Futures  29-Aug-2019          -            0   15543.00   -0.333440   
27  Index Futures  31-Oct-2019          -            0       0.00 -100.000000   
28  Index Futures  26-Sep-2019          -            0       0.00 -100.000000   

    openPrice  highPrice  lowPrice  numberOfContractsTraded  totalTurnover  
0      169.40     169.50    137.65                    26605  110979.607200   
1       66.60      67.50     61.15                    20869   29591.096920   
2      436.00     439.00    420.90                    16967   80552.595750   
3      466.85     467.00    436.25                    16875   60661.722720   
4      111.80     114.00    110.10                     5833   19685.286000   
5      277.60     277.85    275.00                     5355   44992.392000   
6      411.60     413.30    407.25                     4560   30347.780100   
7       66.60      67.40     61.25                     3479    4919.398880   
8       98.00      99.40     96.05                     3287   10310.977600   
9      168.55     168.55    138.20                     3177   13129.984000   
10     138.30     138.85    133.80                     2231    9102.480000   
11      62.20      62.60     60.35                     2181    8124.861600   
12      94.60      94.90     92.50                     1931    8237.600100   
13      89.30      90.00     82.80                     1811    9555.801600   
14      31.30      31.60     30.75                     1559    5930.371200   
15      64.00      64.30     62.35                     1549    6898.260600   
16      54.25      55.00     52.20                     1538    6651.317600   
17       5.50       5.70      5.35                     1212    1871.100000   
18      51.00      53.65     50.90                      894    4260.600000   
19      14.75      15.00     14.55                      292    1976.683500   
20   10910.40   10917.85  10850.30                    39243  328770.326700   
21   10955.00   10958.45  10886.35                     7243   61018.946220   
22   10989.55   10989.55  10931.00                      330    2761.158960   
23   27715.00   27746.05  27505.60                    38495  218171.425716   
24   27810.10   27810.10  27610.70                     2563   14438.541018   
25   27833.15   27847.00  27715.00                      115     638.754620   
26   15550.00   15575.00  15494.00                       36     279.648360   
27       0.00       0.00      0.00                        0       0.000000   
28       0.00       0.00      0.00                        0       0.000000   

    premiumTurnover  openInterest  underlyingValue             timestamp  
0      1.109796e+10         15726           146.50  22-Aug-2019 10:36:21  
1      2.959110e+09         76008            61.95  22-Aug-2019 10:36:21  
2      8.055260e+09         10533           429.80  22-Aug-2019 10:36:21  
3      6.066172e+09         27949           443.95  22-Aug-2019 10:36:21  
4      1.968529e+09         25079           110.40  22-Aug-2019 10:36:21  
5      4.499239e+09         33409           275.75  22-Aug-2019 10:36:21  
6      3.034778e+09         54895           408.90  22-Aug-2019 10:36:21  
7      4.919399e+08         13816            61.95  22-Aug-2019 10:36:21  
8      1.031098e+09         10335            98.40  22-Aug-2019 10:36:21  
9      1.312998e+09          1110           146.50  22-Aug-2019 10:36:21  
10     9.102480e+08         20821           135.00  22-Aug-2019 10:36:21  
11     8.124862e+08         14751            60.55  22-Aug-2019 10:36:21  
12     8.237600e+08         12955            92.75  22-Aug-2019 10:36:21  
13     9.555802e+08          5176            83.00  22-Aug-2019 10:36:21  
14     5.930371e+08          8065            31.40  22-Aug-2019 10:36:21  
15     6.898261e+08         14087            62.60  22-Aug-2019 10:36:21  
16     6.651318e+08          5343            52.45  22-Aug-2019 10:36:21  
17     1.871100e+08         15059             5.60  22-Aug-2019 10:36:21  
18     4.260600e+08          5617            53.45  22-Aug-2019 10:36:21  
19     1.976684e+08          3076            14.75  22-Aug-2019 10:36:21  
20     3.287703e+10        262448         10842.90  22-Aug-2019 10:36:21  
21     6.101895e+09         45640         10842.90  22-Aug-2019 10:36:21  
22     2.761159e+08          3701         10842.90  22-Aug-2019 10:36:21  
23     2.181714e+10         87516         27472.70  22-Aug-2019 10:36:21  
24     1.443854e+09         11998         27472.70  22-Aug-2019 10:36:21  
25     6.387546e+07           614         27472.70  22-Aug-2019 10:36:21  
26     2.796484e+07           207         15539.45  22-Aug-2019 10:36:21  
27     0.000000e+00             0         15539.45  22-Aug-2019 10:36:21  
28     0.000000e+00             5         15539.45  22-Aug-2019 10:36:21  
User contributions licensed under: CC BY-SA
7 People found this is helpful
Advertisement