Your best option is to handle the histogram with numpy like count, index = np.histogram(df['data'], bins=25) , and then use go.Scatter() and set the linetype to horizontal, vertical, horizontal with line=dict(width = 1, shape='hvh'). Take a look at the very last section why go.Histogram() will not be your best option. With a few other specifications for the layout of go.Scatter(), the snippet below will produce the following plot:

Complete code
import plotly.graph_objects as go
import pandas as pd
import numpy as np
import plotly.io as pio
import plotly.express as px
pio.templates.default = "plotly_white"
# random numbers to a df
np.random.seed(12)
df = pd.DataFrame({'data': np.random.randn(500)})
# produce histogram data wiht numpy
count, index = np.histogram(df['data'], bins=25)
# plotly, go.Scatter with line shape set to 'hvh'
fig = go.Figure()
fig.add_traces(go.Scatter(x=index, y = count,
line=dict(width = 1, shape='hvh')))
# y-axis cosmetics
fig.update_yaxes(
showgrid=False,
ticks="inside",
tickson="boundaries",
ticklen=10,
showline=True,
linewidth=1,
linecolor='black',
mirror=True,
zeroline=False)
# x-axis cosmetics
fig.update_xaxes(
showgrid=False,
ticks="inside",
tickson="boundaries",
ticklen=10,
showline=True,
linewidth=1,
linecolor='black',
mirror=True,
zeroline=False)
fig.show()
Why go.Scatter() and not go.Histogram()?
The closest you'll get to your desired plot using your approach with fig = go.Figure(data=[go.Histogram(x=x)]) is this:

And that's pretty close, but you specifically wanted to exclude the vertical lines for each "bar". And I have yet not found a way to exclude or hide them with the go.Histogram setup.
Code for go.Histogram()
import plotly.graph_objects as go
import pandas as pd
import numpy as np
import plotly.io as pio
import plotly.express as px
pio.templates.default = "plotly_white"
import numpy as np
x = np.random.randn(500)
fig = go.Figure(data=[go.Histogram(x=x)])
fig.update_traces(marker=dict(color='rgba(0,0,0,0)', line=dict(width=1, color='blue')))
fig.show()