apache spark connection request in python

Viewed 85

In AWS glue i am trying to get limited data from two tables

how can we get data from relational data from two different tables with a limit

first table i will query only 200 records and using that i need to fetch few column

 table_1 = {  
        "url": url,
        "dbtable": table1,
        "user": db_username,
        "password": db_password,
        "hashexpression": "id > 1 AND id < 200 AND id", ##use the numeric column `id` to read data partitioned.
        "hashpartitions": 7 ##number of parallel reads of the JDBC table
    }

datasource0 = glueContext.create_dynamic_frame_from_options(
    connection_type="postgresql", 
    connection_options=table_1,
    transformation_ctx = "datasource0"
    );

table_2 = {  
        "url": url,
        "dbtable": table2,
        "user": db_username,
        "password": db_password,
        "hashexpression": "id", ##something here  to read only related data.
        "hashpartitions": 7 
    }
0 Answers
Related