Lets suppose that for Dimension1 ,attribute1, and the attribute "Value 1", you had nothing in Fact, so this is removed from the result, now you forced Excel to display it by selecting the option. When you add another dimension's attribute lets say Dimension2.Attribute1, since "Value 1" had nothing in Fact, therefore Cube will not understand which value of of Dimension2.Attribute1 is to be displayed in front of Dimension1.Attribute1, therefore it will display all its values. So if we have 3 values in Dimension2, attribute 1 then "Value 1" will be repeated three times. Now with Excel you cannot solve the issue, however it might just be possible to write an MDX query that works.
Edit: Query Added.
The below sample query is based on AdventureWorks, the first sample shows that the result has some nulls, if i un-comment the "non-empty" all null values will vanish, go ahead try it.
select [Measures].[Internet Sales Amount] on columns,
--non empty
[Product].[Subcategory].[Subcategory]
on rows
from
[Adventure Works]
Result without non empty

Now lets add another dimension to the query. Notice that the null value for the first row(Bib-shorts) is now repeated for all values of the second dimension, Since cube has no way to determine which value to display.
select [Measures].[Internet Sales Amount] on columns,
--non empty
([Product].[Subcategory].[Subcategory],[Date].[Calendar Quarter of Year].[Calendar Quarter of Year])
on rows
from
[Adventure Works]
Result

Now the above result shows the issue you are facing. What we now need to do is whenever there is a null value we dont need the individual members of the second dimension, rather a place holder to satisfy the tuple, will work.
In the query below I have two tuples
1) for the not null data-points. Here we display the actual member of the second dimension.
2) for the null data-points, here we use ".defaultmember" which basically means that the second dimension will behave as it was never selected. Have a close look at the second dimension it says "All Period"
select [Measures].[Internet Sales Amount] on columns,
--non empty
{filter(([Product].[Subcategory].[Subcategory],[Date].[Calendar Quarter of Year].[Calendar Quarter of Year]),[Measures].[Internet Sales Amount]>0),
filter(([Product].[Subcategory].[Subcategory],[Date].[Calendar Quarter of Year].defaultmember),[Measures].[Internet Sales Amount]=null)
}
on rows
from
[Adventure Works]
Result:
