For an Amplify Datastore setup, is there a way to turn off AWS AppSync to just work locally?

Yes! DataStore can be used with or without any cloud synchronization.

Android

Cloud sync is enabled when you add an API plugin. To disable cloud sync, don't add AWSApiPlugin():

Amplify.addPlugin(AWSDataStorePlugin())

// Comment out this line:
// Amplify.addPlugin(AWSApiPlugin())

Amplify.configure(applicationContext)

iOS

Sync is enabled when there is an API plugin present. To disable sync, do not add AWSAPIPlugin():

let dataStorePlugin =
    AWSDataStorePlugin(modelRegistration: AmplifyModels())
try Amplify.add(plugin: dataStorePlugin)

// Comment out this line:
// try Amplify.add(plugin: AWSAPIPlugin())

try Amplify.configure()

JavaScript

The approach is slightly different. Look for any API configurations in the aws-exports.js file, and remove them. Specifically, you should remove any config entries with aws_appsync_* in the prefix:

const awsmobile = {
    // ...
    "aws_appsync_graphqlEndpoint": "https://yourid.appsync-api.us-east-1.amazonaws.com/graphql",
    "aws_appsync_region": "us-east-1",
    "aws_appsync_authenticationType": "AMAZON_COGNITO_USER_POOLS",
    // ...
};

You might take a look on the Full offline option using AWS DataStore and then allow an optional activation for the the cloud sync features in Android

Seems like DataStore mind set is offline first with API available in AWS, so there is no option to not create the API in the AWS while performing amplify push.

The normal solution to working offline during development is to not run amplify push during project setup. You can run your app locally and everything works normally, but there is no attempt to sync to the cloud. When you change your schema you run amplify codegen models and all the models are updated locally.

When you are finally ready to deploy to the cloud and start running in online/offline mode you run amplify push to provision all the cloud resources. Even after this you can still run local only by using the browser dev tools to disable the network connection.

I don't know how to set the project back to a pre amplify push state, but I would like the answer. I'm sure it is a setting somewhere in the amplify configuration in the project.