The only overt issue I see with the code is that the route rendering on path="/institute/*" is wrapping several other routes. When Route components are not directly rendered by a Switch component so they are exclusively matched and rendered, then they work like normal when rendered by a router, they inclusively render all matches.
I don't see any issue when the user.role is "student" and all routes are accessible. The issue I see is when user.role is "teacher".
Example:
Assume user = { role: "teacher" } and the URL path is "/institue/lectures"
<Switch>
<Route path="/institute/*">
{user.role === "teacher" || user.role === "student" ? (
<Route path="/institute/">
<Route exact path="/institute/lectures" component={Lectures} /> // (1)*
<Route exact path="/institute/modules" component={Modules} />
</Route>
) : (
<Redirect to="/" />
)}
{user.role === "student" ? (
<Route exact path="/institute/student" component={Student} />
) : (
<Redirect to="/" /> // (2)**
)}
</Route>
...
</Switch>
The path="/institute/*" route matches and returns its content which consists of several nested routes that will now inclusively match. user.role is "teacher" so the path="/institute/" route renders its content, then path="/institute/lectures" is also matched and renders its content, the Lectures component, #1.
Since user.role === "student" evaluates false, the Redirect component renders and when it does, it unconditionally redirects to the home "/" path, #2.
This will occur for any user with role !== "student".
Adding another Switch component inside <Route path="/institute/*"> route will fix this as now only the first matching Route or Redirect will be matched and rendered, exclusively.
<Switch>
<Route path="/institute/*">
<Switch>
{user.role === "teacher" || user.role === "student" ? (
<Route path="/institute/">
<Route exact path="/institute/lectures" component={Lectures} />
<Route exact path="/institute/modules" component={Modules} />
</Route>
) : (
<Redirect to="/" />
)}
{user.role === "student" ? (
<Route exact path="/institute/student" component={Student} />
) : (
<Redirect to="/" />
)}
</Switch>
</Route>
...
</Switch>

This code is a little convoluted and could be simplified a bit. Create a custom route component to handle checking an access role and redirecting.
const RoleRoute = ({ roles = [], user = {}, ...props }) =>
!roles.length || roles.includes(user.role) ? (
<Route {...props} />
) : (
<Redirect to="/" />
);
Render the routes directly into the Switch.
<Switch>
<RoleRoute
roles={["teacher", "student"]}
user={user}
path="/institute/lectures"
component={Lectures}
/>
<RoleRoute
roles={["teacher", "student"]}
user={user}
path="/institute/modules"
component={Modules}
/>
<RoleRoute
roles={["student"]}
user={user}
path="/institute/student"
component={Student}
/>
...
</Switch>
