Why you want React-Router for this?
Can't you directly wrap the buttons into <a> tag and provide a href attribute to <a> tag with the id of section where you want to scroll.
e.g.
<a href="#home"><button>Home</button></a>
Somewhere in you component,
<div id="home">
...
</div>
For smooth scrolling effect you can add this css,
html{scroll-behavior:smooth}
Updated
On the basis of comment, you can make use of react-router-hash-link.
yarn add react-router-hash-link
or
npm install --save react-router-hash-link
// In YourComponent.js
...
import { HashLink as Link } from 'react-router-hash-link';
...
// Use it just like a RRv4 (to can be a string or an object, see RRv4 api for details):
<Link to="/some/path#with-hash-fragment">Link to Hash Fragment</Link>