I need to execute 2 drool rules in Java with different logic having dependency between them, used ruleflow-group but 2nd rule not getting executed

You're not firing either rule -- int y holds the total number of rules that are fired, and neither does. The ruleflow-group is an unnecessary complication.

The reason is simple: both rules require that there be a OutputDTO instance in working memory, and you don't have any. You only insert an instance of A.

If you add this line, your rules will fire (based on the values present in A):

kieSession.insert(new OutputDTO());

However, I don't think your rules do what you want them to do. I think what you want to do is, when each rule fires, to add a new OutputDTO into the outputDTOs global.

To do this, you'll want to new up an instance of OutputDTO in each 'then' clause, set the values as you need, and add that to the global.

rule "ABC with Approved Status"
ruleflow-group "status"
when
  $transaction: A( $b : b != null )
  exists( B( group == "ABC" , status == "Approved" )  from $b )           
then
  OutputDTO $outputDTO = new OutputDTO();
  $outputDTO.setGroup("ABC");
  $outputDTO.setStatus("Approved");
  outputDTOs.add($outputDTO);
end

rule "XYZ with Approved Status"
ruleflow-group "status"
when
  $transaction: A( $b : b != null )
  exists( B( group == "XYZ" , status == "Approved" )  from $b )     
then
  OutputDTO $outputDTO = new OutputDTO();     
  $outputDTO.setGroup("XYZ");
  $outputDTO.setStatus("Approved");
  outputDTOs.add($outputDTO);
end

What you're doing in the rules you originally posted, is checking for the presence of an existing OutputDTO object in working memory, changing its values, and updating the global. As written, if both rules hit, you'll likely have two of the same object reference in the list with the same values which will match whichever rule evaluated last.

This all being said, your question title says your rules have a "dependency between them". As written, they don't. They're perfectly fine, completely independent rules.

You also state that you need the ABC rule to fire before the XYZ rule, which is why you used ruleflow-groups. This is equally unnecessary; if you must have order, you can use salience -- though this isn't necessarily good design. (But the option is there, so you might as well use it especially since your question neglects to explain why ABC must occur before XYZ, nor now the two rules are "dependent".)

_{Further, your code doesn't show how you're actually doing this. In it, you fire a "scope" agenda-group, followed by a "status" agenda-group; assuming you're aware that agenda groups and ruleflow groups are merged in modern drools, neither of your rules is part of the "scope" group....}

Here's an example for how you'd have rule ABC fire before XYZ without bothering with the over-complication of ruleflow-groups.

rule "ABC with Approved Status"
salience 1
when
  $transaction: A( $b : b != null )
  exists( B( group == "ABC" , status == "Approved" )  from $b )           
then
  OutputDTO $outputDTO = new OutputDTO();
  $outputDTO.setGroup("ABC");
  $outputDTO.setStatus("Approved");
  outputDTOs.add($outputDTO);
end

rule "XYZ with Approved Status"
when
  $transaction: A( $b : b != null )
  exists( B( group == "XYZ" , status == "Approved" )  from $b )     
then
  OutputDTO $outputDTO = new OutputDTO();     
  $outputDTO.setGroup("XYZ");
  $outputDTO.setStatus("Approved");
  outputDTOs.add($outputDTO);
end

The salience 1 rule will be evaluated before the rule without salience. (Default salience, if not specified, is 0; note that you can have negative saliences as well. It's very similar to the z-index in CSS, if you're familiar.)

Then, of course, you could simply fire the rules like:

List<OutputDTO> outputDTOs = new ArrayList<>();
kieSession.setGlobal("outputDTOs", outputDTOs);
kieSession.insert(a); // A object
kieSession.fireAllRules();
kieSession.dispose();
return outputDTOs;