Concat two array in a specific condition?

user_15765347 0
Viewed 92

I require to concat two arrays of unequal size:

Array-1:

A = ["year","month","day","hour","minute","second", "a", "b", "c", "d"]
data1 = pd.read_csv('event_5.txt',sep='\t',names=A)
array1=data1[['year', 'month', 'day']]

Array-2:

B=["station", "phase", "hour", "minute", "second"]
arr_data = pd.read_csv('arrival_5.txt',sep='\t',names=B)
ar_t= arr_data[['hour', 'minute', 'second']]
array2 = pd.DataFrame(ar_t)

The required output is shown below: here, [2019 11 9] is the array-1 reshaped to match the dimensions of the second array and then concat. However, in the case of reshaping, I need to check the dimensions of the second array every time. Therefore, I need an automated script that can concat unequal arrays.

Array-1: The first array always have the same dimensions

        year  month  day
     0  2019     11    9

Array-2: Variable dimension columns are fixed but rows change for each iteration:

    hour  minute  second
0     14      57   41.80
1     14      58    3.47
2     14      57   25.99
3     14      57   37.00
4     14      57   29.86
5     14      57   40.24
6     14      57   32.61
7     14      57   42.26
8     14      57   29.74
9     14      57   42.36
10    14      57   46.00
11    14      58    8.69
12    14      57   34.50
13    14      57   48.97
14    14      57   30.30
15    14      57   39.78
16    14      57   32.45
17    14      57   47.83
18    14      57   25.86
19    14      57   36.30
20    14      57   17.90
21    14      57   23.40
22    14      57   34.64
23    14      57   50.95
24    14      57   35.90
25    14      57   50.64

Required output:

  Year  month  day  hour  minute  second
0   2019     11    9    14      57   41.80
1   2019     11    9    14      58    3.47
2   2019     11    9    14      57   25.99
3   2019     11    9    14      57   37.00
4   2019     11    9    14      57   29.86
5   2019     11    9    14      57   40.24
6   2019     11    9    14      57   32.61
7   2019     11    9    14      57   42.26
8   2019     11    9    14      57   29.74
9   2019     11    9    14      57   42.36
10  2019     11    9    14      57   46.00
11  2019     11    9    14      58    8.69
12  2019     11    9    14      57   34.50
13  2019     11    9    14      57   48.97
14  2019     11    9    14      57   30.30
15  2019     11    9    14      57   39.78
16  2019     11    9    14      57   32.45
17  2019     11    9    14      57   47.83
18  2019     11    9    14      57   25.86
19  2019     11    9    14      57   36.30
20  2019     11    9    14      57   17.90
21  2019     11    9    14      57   23.40
22  2019     11    9    14      57   34.64
23  2019     11    9    14      57   50.95
24  2019     11    9    14      57   35.90
25  2019     11    9    14      57   50.64
python
3 Answers

Assigning a constant value to a DataFrame column

If your first array is always a single-row dataframe, or a monodimensional array, then you can just use pandas to assign a constant value to a column.

The syntax is my_dataframe["new_column"] = constant_value.

Because arr1 is a DataFrame, accessing a column will give us a Series. To get its constant value, then, we need to take the value in cell indexed by 0 - or the first row.

In your case this becomes:

>>> type(arr1), type(arr2)
(pandas.core.frame.DataFrame, pandas.core.frame.DataFrame)
>>> arr2["year"] = arr1["year"].loc[0]
>>> arr2["month"] = arr1["month"].loc[0]
>>> arr2["day"] = arr1["day"].loc[0]
>>> arr2
    hours  minutes    seconds  year  month  day
0       9        6  22.001464  2019     11    9
1       8       21  28.412044  2019     11    9
2      10        7  22.433552  2019     11    9
3      18       37  19.551359  2019     11    9
4      19        1  40.722019  2019     11    9
..    ...      ...        ...   ...    ...  ...
95      2       16  48.368643  2019     11    9
96     19       22  25.034936  2019     11    9
97     10        0  20.163870  2019     11    9
98     16       35  27.251357  2019     11    9
99      8       26  54.200897  2019     11    9

Remember that this will work in-place, modifying arr2 object.

Accessing the numpy array behind the DataFrame

If you need the multidimensional array, you can just call:

>>> arr2_np = arr2.to_numpy()

Sorting columns based on your use-case

If you need to sort the columns, you can just take a different view of them, like this:

>>> cols = arr2.columns.to_list()
>>> cols2 = cols[3:] + cols[:3]
>>> arr2[cols2] 
    year  month  day  hours  minutes    seconds
0   2019     11    9      9        6  22.001464
1   2019     11    9      8       21  28.412044
2   2019     11    9     10        7  22.433552
3   2019     11    9     18       37  19.551359
4   2019     11    9     19        1  40.722019
..   ...    ...  ...    ...      ...        ...
95  2019     11    9      2       16  48.368643
96  2019     11    9     19       22  25.034936
97  2019     11    9     10        0  20.163870
98  2019     11    9     16       35  27.251357
99  2019     11    9      8       26  54.200897
user_116470250

You can use numpy.column_stack:

np.column_stack((array_1, array_2))

a
#array([[0, 1, 2],
#       [3, 4, 5]])

b
#array([0, 1])

np.column_stack((a, b))
#array([[0, 1, 2, 0],
#       [3, 4, 5, 1]])
user_50842320

this worked for me:

import numpy as np

arr1=[2019, 12, 17]
arr2=[12, 34, 17,
      18, 17, 36, 
      15, 23, 40]

print(arr1,arr2)

output:

[2019, 12, 17] [12, 34, 17, 18, 17, 36, 15, 23, 40]

arr2 = np.array(arr2).reshape((3,3))
arr1 = np.array([arr1,]*3)
newArray = np.hstack((arr1,arr2))

output:

 array([[2019,   12,   17,   12,   34,   17],
       [2019,   12,   17,   18,   17,   36],
       [2019,   12,   17,   15,   23,   40]])

update, to increase performance for large datasets, simple only stack new value after the array is once reshaped:

arr1=[2019, 12, 17]
newEntry = [1,2,3]
nE = np.hstack((arr1,newEntry))
np.vstack((newArray,nE))

output:

array([[2019,   12,   17,   12,   34,   17],
       [2019,   12,   17,   18,   17,   36],
       [2019,   12,   17,   15,   23,   40],
       [2019,   12,   17,    1,    2,    3]])

update without knowledge of exakt dimension you can simply use:

np.arange(arr2).reshape(-1, 3)
user_41576380
Related