Constructing a randomised matrix with no duplicates but fixed partial input

I´m facing a problem with constructing a randomised matrix where I partially already have values (that need to stay fixed - so no further randomisation there).

Lets see:

matrix should end up being 10 by 10

 n <- 10 

I do want my first rows to be the data I enter. e.g:

  row1<- c(1,4,7,6,5,3,2,8,9,10)
  row2<- c(10,7,3,2,1,4,5,9,8,6)
  row3<- c(9,2,4,3,8,7,10,1,6,5)

To bild a matrix with 10 rows (and 10 columns) I combined those rows with samples (no replace because I want each number to be unique in each row).

 first.rows<-rbind(row1,row2,row3,sample(n,n,replace=F),sample(n,n,replace=F),sample(n,n,replace=F),sample(n,n,replace=F),sample(n,n,replace=F),sample(n,n,replace=F),sample(n,n,replace=F))

output:

     [,1] [,2] [,3] [,4] [,5] [,6] [,7] [,8] [,9] [,10]
row1    1    4    7    6    5    3    2    8    9    10
row2   10    7    3    2    1    4    5    9    8     6
row3    9    2    4    3    8    7   10    1    6     5
        6    1    5    4    2   10    3    8    7     9
        2    5    7    8    9    6    1    3    4    10
       10    6    4    1    8    3    7    2    5     9
        8    5    3    2    4    1   10    7    6     9
       10    7    9    6    8    2    5    4    3     1
        1   10    8    4    7    3    5    2    6     9
        2    1   10    4    8    9    3    6    5     7

So far so good.. However now I have the problem that there is no control for unique numbers in the columns. This is what I would need though. I get that this the case because I used rbind (and therefore only the function of no duplicates only works for the rows). But I do not know how else to approach this problem. The rows 1-3 should stay exactly as they are now.

Any ideas?