WebView How to run even when app is in background/closed (foreground service active)

user_9729549 0
Viewed 2051

I'm building an app which will scrape some data from a website and shows a notification when some criteria are met.

Everything works well without problems when the app is open (because the WebView is being rendered) but when I close the app the WebView is disabled so I cannot use it to scrape data anymore.

The scraping code is inside a class called from a ForegroundService.

I've already looked on the internet but I'm unable to find a solution or a substitute to WebView, do you have any ideas?


I'm sorry if this question looks stupid to you, I've started to develop for mobile just one week ago


Below the JDMonitoring class which is called from the AlarmTask class

using System;
using System.Collections.Generic;
using System.Net;
using System.Threading.Tasks;
using Xamarin.Forms;

namespace CGSJDSportsNotification {
    public class JDMonitoring {
        class Ticket {
            string owner;
            string title;
            string store;
            string lastUpdated;
            string link;

            public string ID { get; set; }
            public string Owner {
                get {
                    return owner == null ? "Nobody" : owner;
                } set {
                    owner = value.Remove(0, value.IndexOf('(') + 1).Replace(")", "");
                }
            }
            public string Title { 
                get {
                    return title;
                } set {
                    if (value.StartsWith("(P"))
                        title = value.Remove(0, value.IndexOf(')') + 2);
                }
            }
            public string Status { get; set; }
            public string Store { 
                get {
                    return store;
                } set {
                    store = value.Replace(@"\u003C", "").Replace(">", "");
                } 
            }
            public string LastUpdated { 
                get {
                    return lastUpdated;
                } set {
                    string v;

                    int time = Convert.ToInt32(System.Text.RegularExpressions.Regex.Replace(value, @"[^\d]+", ""));

                    // Convert to minutes
                    if (value.Contains("hours"))
                        time *= 60;

                    v = time.ToString();

                    if (value.Contains("seconds"))
                        v = v.Insert(v.Length, " sec. ago");
                    else
                        v = v.Insert(v.Length, " min. ago");

                    lastUpdated = v;
                } 
            }
            public string Link { 
                get {
                    return link;
                } set {
                    link = "https://support.jdplc.com/" + value;
                } 
            }
        }

        public JDMonitoring() {
            WB.Source = JDQueueMainUrl;
            WB.Navigated += new EventHandler<WebNavigatedEventArgs>(OnNavigate);
        }

        IForegroundService FgService { get { return DependencyService.Get<IForegroundService>(); } }

        WebView WB { get; } = MainPage.UI.MonitoringWebView;
        string JDQueueMainUrl { get; } = "https://support.jdplc.com/rt4/Search/Results.html?Format=%27%3Cb%3E%3Ca%20href%3D%22__WebPath__%2FTicket%2FDisplay.html%3Fid%3D__id__%22%3E__id__%3C%2Fa%3E%3C%2Fb%3E%2FTITLE%3A%23%27%2C%0A%27%3Cb%3E%3Ca%20href%3D%22__WebPath__%2FTicket%2FDisplay.html%3Fid%3D__id__%22%3E__Subject__%3C%2Fa%3E%3C%2Fb%3E%2FTITLE%3ASubject%27%2C%0AStatus%2C%0AQueueName%2C%0AOwner%2C%0APriority%2C%0A%27__NEWLINE__%27%2C%0A%27__NBSP__%27%2C%0A%27%3Csmall%3E__Requestors__%3C%2Fsmall%3E%27%2C%0A%27%3Csmall%3E__CreatedRelative__%3C%2Fsmall%3E%27%2C%0A%27%3Csmall%3E__ToldRelative__%3C%2Fsmall%3E%27%2C%0A%27%3Csmall%3E__LastUpdatedRelative__%3C%2Fsmall%3E%27%2C%0A%27%3Csmall%3E__TimeLeft__%3C%2Fsmall%3E%27&Order=DESC%7CASC%7CASC%7CASC&OrderBy=LastUpdated%7C%7C%7C&Query=Queue%20%3D%20%27Service%20Desk%20-%20CGS%27%20AND%20(%20%20Status%20%3D%20%27new%27%20OR%20Status%20%3D%20%27open%27%20OR%20Status%20%3D%20%27stalled%27%20OR%20Status%20%3D%20%27deferred%27%20OR%20Status%20%3D%20%27open%20-%20awaiting%20requestor%27%20OR%20Status%20%3D%20%27open%20-%20awaiting%20third%20party%27%20)&RowsPerPage=0&SavedChartSearchId=new&SavedSearchId=new";
        bool MonitoringIsInProgress { get; set; } = false;

        public bool IsConnectionAvailable {
            get {
                try {
                    using (new WebClient().OpenRead("http://google.com/generate_204"))
                        return true;
                } catch {
                    return false;
                }
            }
        }

        async Task<bool> IsOnLoginPage() {
            if (await WB.EvaluateJavaScriptAsync("document.getElementsByClassName('left')[0].innerText") != null)
                return true;

            return false;
        }

        async Task<bool> Login() {
            await WB.EvaluateJavaScriptAsync($"document.getElementsByName('user')[0].value = '{UserSettings.SecureEntries.Get("rtUser")}'");
            await WB.EvaluateJavaScriptAsync($"document.getElementsByName('pass')[0].value = '{UserSettings.SecureEntries.Get("rtPass")}'");

            await WB.EvaluateJavaScriptAsync("document.getElementsByClassName('button')[0].click()");

            await Task.Delay(1000);

            // Checks for wrong credentials error
            if (await WB.EvaluateJavaScriptAsync("document.getElementsByClassName('action-results')[0].innerText") == null)
                return true;

            return false;
        }

        async Task<List<Ticket>> GetTickets() {
            List<Ticket> tkts = new List<Ticket>();

            // Queue tkts index (multiple of 2)
            int index = 2;

            // Iterates all the queue
            while (await WB.EvaluateJavaScriptAsync($"document.getElementsByClassName('ticket-list collection-as-table')[0].getElementsByTagName('tbody')[0].getElementsByTagName('tr')[{index}].innerText") != null) {
                Ticket tkt = new Ticket();

                tkt.LastUpdated = await WB.EvaluateJavaScriptAsync($"document.getElementsByClassName('ticket-list collection-as-table')[0].getElementsByTagName('tbody')[0].getElementsByTagName('tr')[{index + 1}].getElementsByTagName('td')[4].innerText");

                // Gets only the tkts which are not older than the value selected by the user
                if (Convert.ToInt32(System.Text.RegularExpressions.Regex.Replace(tkt.LastUpdated, @"[^\d]+", "")) > Convert.ToInt32(UserSettings.Entries.Get("searchTimeframe")))
                    break;

                tkt.ID     = await WB.EvaluateJavaScriptAsync($"document.getElementsByClassName('ticket-list collection-as-table')[0].getElementsByTagName('tbody')[0].getElementsByTagName('tr')[{index}].getElementsByTagName('td')[0].innerText");
                tkt.Owner  = await WB.EvaluateJavaScriptAsync($"document.getElementsByClassName('ticket-list collection-as-table')[0].getElementsByTagName('tbody')[0].getElementsByTagName('tr')[{index}].getElementsByTagName('td')[4].innerText");
                tkt.Title  = await WB.EvaluateJavaScriptAsync($"document.getElementsByClassName('ticket-list collection-as-table')[0].getElementsByTagName('tbody')[0].getElementsByTagName('tr')[{index}].getElementsByTagName('td')[1].innerText");
                tkt.Status = await WB.EvaluateJavaScriptAsync($"document.getElementsByClassName('ticket-list collection-as-table')[0].getElementsByTagName('tbody')[0].getElementsByTagName('tr')[{index}].getElementsByTagName('td')[2].innerText");
                tkt.Store  = await WB.EvaluateJavaScriptAsync($"document.getElementsByClassName('ticket-list collection-as-table')[0].getElementsByTagName('tbody')[0].getElementsByTagName('tr')[{index + 1}].getElementsByTagName('td')[1].innerText");
                tkt.Link   = await WB.EvaluateJavaScriptAsync($"document.getElementsByClassName('ticket-list collection-as-table')[0].getElementsByTagName('tbody')[0].getElementsByTagName('tr')[{index}].getElementsByTagName('td')[1].getElementsByTagName('a')[0].getAttribute('href')");

                tkts.Add(tkt);
                index += 2;
            }

            return tkts;
        }

        //async Task<string> QueueGetTkt

        async void OnNavigate(object sender, WebNavigatedEventArgs args) {
            if (MonitoringIsInProgress)
                return;

            if (IsConnectionAvailable) {
                if (await IsOnLoginPage()) {
                    if (await Login() == false) {
                        // If the log-in failed we can't proceed
                        MonitoringIsInProgress = false;

                        FgService.NotificationNewTicket("Log-in failed!", "Please check your credentials");

                        // Used to avoid an infinite loop of OnNavigate method calls
                        WB.Source = "about:blank";
                        return;
                    }
                }

                // Main core of the monitoring
                List<Ticket> tkts = await GetTickets();

                if (tkts.Count > 0) {
                    foreach(Ticket t in tkts) {
                        // Looks only after the tkts with the country selected by the user (and if it was selected by the user, also for the tkts without a visible country)

                        // Firstly we look in the title
                        if (t.Title.Contains(MainPage.UI.CountryPicker.SelectedItem.ToString())) {
                            FgService.NotificationNewTicket($"[{t.ID}] {t.LastUpdated}",
                                $"{t.Title}\r\n\r\n" +
                                $"Status:             {t.Status}\r\n" +
                                $"Owner:             {t.Owner}\r\n" +
                                $"Last updated: {t.LastUpdated}");

                            break;
                        }
                    }
                }
            }


            MonitoringIsInProgress = false;
        }
    }
}

AlarmTask class

using Android.App;
using Android.Content;
using Android.Support.V4.App;

namespace CGSJDSportsNotification.Droid {
    [BroadcastReceiver(Enabled = true, Exported = true, DirectBootAware = true)]
    [IntentFilter(new string[] { Intent.ActionBootCompleted, Intent.ActionLockedBootCompleted, "android.intent.action.QUICKBOOT_POWERON", "com.htc.intent.action.QUICKBOOT_POWERON" }, Priority = (int)IntentFilterPriority.HighPriority)]
    public class AlarmTask : BroadcastReceiver {
        IAlarm _MainActivity { get { return Xamarin.Forms.DependencyService.Get<IAlarm>(); } }

        public override void OnReceive(Context context, Intent intent) {
            if (intent.Action != null) {
                if (intent.Action.Equals(Intent.ActionBootCompleted)) {
                    // Starts the app after reboot
                    var serviceIntent = new Intent(context, typeof(MainActivity));
                    serviceIntent.AddFlags(ActivityFlags.NewTask);
                    context.StartActivity(serviceIntent);

                    Intent main = new Intent(Intent.ActionMain);
                    main.AddCategory(Intent.CategoryHome);
                    context.StartActivity(main);

                    // Does not work, app crashes on boot received
                    /*if (UserSettings.Entries.Exists("monitoringIsRunning")) {
                        if ((bool)UserSettings.Entries.Get("monitoringIsRunning"))
                            FgService.Start();
                    }*/
                }
            } else
                // Checks for new tkts on a new thread
                new JDMonitoring();
                // Restarts the alarm
                _MainActivity.AlarmStart();
        }

        // Called from JDMonitoring class
        public static void NotificationNewTicket(string title, string message, bool icoUnknownCountry = false) {
            new AlarmTask().NotificationShow(title, message, icoUnknownCountry);
        }

        void NotificationShow(string title, string message, bool icoUnknownCountry) {
            int countryFlag = Resource.Drawable.newTktUnknownCountry;

            if (icoUnknownCountry == false) {
                switch (MainPage.UI.CountryPicker.SelectedItem.ToString()) {
                    case "Italy":
                        countryFlag = Resource.Drawable.newTktItaly;
                        break;
                    case "Spain":
                        countryFlag = Resource.Drawable.newTktSpain;
                        break;
                    case "Germany":
                        countryFlag = Resource.Drawable.newTktGermany;
                        break;
                    case "Portugal":
                        countryFlag = Resource.Drawable.newTktPortugal;
                        break;
                }
            }

            var _intent = new Intent(Application.Context, typeof(MainActivity));
            _intent.AddFlags(ActivityFlags.ClearTop);
            _intent.PutExtra("jdqueue_notification", "extra");
            var pendingIntent = PendingIntent.GetActivity(Application.Context, 0, _intent, PendingIntentFlags.OneShot);


            NotificationCompat.Builder notificationBuilder = new NotificationCompat.Builder(Application.Context, "newTktNotification_channel")
                    .SetVisibility((int)NotificationVisibility.Public)
                    .SetPriority((int)NotificationPriority.High)
                    .SetDefaults((int)NotificationDefaults.Sound | (int)NotificationDefaults.Vibrate | (int)NotificationDefaults.Lights)

                    .SetSmallIcon(Resource.Drawable.newTktNotification)
                    .SetLargeIcon(Android.Graphics.BitmapFactory.DecodeResource(Application.Context.Resources, countryFlag))

                    .SetSubText("Click to check the queue")
                    .SetStyle(new NotificationCompat.BigTextStyle()
                        .SetBigContentTitle("New ticket available!")
                        .BigText(message))
                    .SetContentText(title)


                    .SetAutoCancel(true)
                    .SetContentIntent(pendingIntent);

            NotificationManagerCompat.From(Application.Context).Notify(0, notificationBuilder.Build());
        }
    }
}

And the ForegroundService class which is responsible to trigger for the first time the alarm

using Android.App;
using Android.Content;
using Android.OS;

namespace CGSJDSportsNotification.Droid {
    [Service]
    class ForegroundService : Service {
        IAlarm _MainActivity { get { return Xamarin.Forms.DependencyService.Get<IAlarm>(); } }

        public override IBinder OnBind(Intent intent) { return null; }

        public override StartCommandResult OnStartCommand(Intent intent, StartCommandFlags flags, int startId) {
            // Starts the Foreground Service and the notification channel
            StartForeground(9869, new ForegroundServiceNotification().ReturnNotif());

            Android.Widget.Toast.MakeText(Application.Context, "JD Queue - Monitoring started!", Android.Widget.ToastLength.Long).Show();

            _MainActivity.AlarmStart();

            return StartCommandResult.Sticky;
        }

        public override void OnDestroy() {
            Android.Widget.Toast.MakeText(Application.Context, "JD Queue - Monitoring stopped!", Android.Widget.ToastLength.Long).Show();

            _MainActivity.AlarmStop();

            UserSettings.Entries.AddOrEdit("monitoringIsRunning", false);
            UserSettings.Entries.AddOrEdit("monitoringStopPending", false, false);

            base.OnDestroy();
        }

        public override bool StopService(Intent name) {
            return base.StopService(name);
        }
    }
}



Thank you!

xamarin.android android-webview c#
1 Answers

[BETTER-FINAL-SOLUTION]
After several hours I've discovered Android WebView which does exactly what I need (I'm developing this app only for Android)

I've written this Browser helper class

class Browser {
    public Android.Webkit.WebView WB;
    static string JSResult;

    public class CustomWebViewClient : WebViewClient {
        public event EventHandler<bool> OnPageLoaded;

        public override void OnPageFinished(Android.Webkit.WebView view, string url) {
            OnPageLoaded?.Invoke(this, true);
        }
    }

    public Browser(CustomWebViewClient wc, string url = "") {
        WB = new Android.Webkit.WebView(Android.App.Application.Context);
        WB.Settings.JavaScriptEnabled = true;


        WB.SetWebViewClient(wc);
        WB.LoadUrl(url);
    }

    public string EvalJS(string js) {
        JSInterface jsi = new JSInterface();

        WB.EvaluateJavascript($"javascript:(function() {{ return {js}; }})()", jsi);

        return JSResult;
    }

    class JSInterface : Java.Lang.Object, IValueCallback {
        public void OnReceiveValue(Java.Lang.Object value) {
            JSResult = value.ToString();
        }
    }
}


[EDIT]
Improved the JS returning function with async callbacks (so the JS return value will be always delivered).

Credits to ChristineZuckerman 

class Browser {
    public Android.Webkit.WebView WB;

    public class CustomWebViewClient : WebViewClient {
        public event EventHandler<bool> OnPageLoaded;

        public override void OnPageFinished(Android.Webkit.WebView view, string url) {
            OnPageLoaded?.Invoke(this, true);
        }
    }

    public Browser(CustomWebViewClient wc, string url = "") {
        WB = new Android.Webkit.WebView(Android.App.Application.Context);
        WB.ClearCache(true);
        WB.Settings.JavaScriptEnabled = true;
        WB.Settings.CacheMode = CacheModes.NoCache;
        WB.Settings.DomStorageEnabled = true;
        WB.Settings.SetAppCacheEnabled(false);
        WB.Settings.UserAgentString = "Mozilla/5.0 (Windows; U; Windows NT 6.1; en-US) AppleWebKit/534.10 (KHTML, like Gecko) Chrome/8.0.552.224 Safari/534.10";
        WB.LoadUrl(url);

        WB.SetWebViewClient(wc);
    }

    public async Task<string> EvalJS(string js, bool returnNullObjectWhenNull = true) {
        string JSResult = "";
        ManualResetEvent reset = new ManualResetEvent(false);

        Device.BeginInvokeOnMainThread(() => {
            WB?.EvaluateJavascript($"javascript:(function() {{ return {js}; }})()", new JSInterface((r) => {
                JSResult = r;
                reset.Set();
            }));
        });

        await Task.Run(() => { reset.WaitOne(); });
        return JSResult == "null" ? returnNullObjectWhenNull ? null : "null" : JSResult;
    }

    class JSInterface : Java.Lang.Object, IValueCallback {
        private Action<string> _callback;

        public JSInterface(Action<string> callback) {
            _callback = callback;
        }

        public void OnReceiveValue(Java.Lang.Object value) {
            string v = value.ToString();

            if (v.StartsWith('"') && v.EndsWith('"'))
                v = v.Remove(0, 1).Remove(v.Length - 2, 1);

            _callback?.Invoke(v);
        }
    }
}



Example:

Browser.CustomWebViewClient wc = new Browser.CustomWebViewClient();
wc.OnPageLoaded += BrowserOnPageLoad;

Browser browser = new Browser(wc, "https://www.google.com/");

void BrowserOnPageLoad(object sender, bool e) {
    string test = browser.EvalJS("document.getElementsByClassName('Q8LRLc')[0].innerText");

    // 'test' will contain the value returned from the JS script
    // You can acces the real WebView object by using
    // browser.WB
}

// OR WITH THE NEW RETURNING FUNCTION

async void BrowserOnPageLoad(object sender, bool e) {
    string test = await browser.EvalJS("document.getElementsByClassName('Q8LRLc')[0].innerText");

    // 'test' will contain the value returned from the JS script
    // You can acces the real WebView object by using
    // browser.WB
}

[FINAL-SOLUTION]
Finally I've found an easy and efficient alternative to WebView.
Now I'm using SimpleBroswer and works great!


[SEMI-SOLUTION]
Alright, I've written a workaround but I don't really like this idea, so please, if you know a better method let me know.


Below my workaround:

In my ForegroundServiceHelper interface I've added a method to check if the MainActivity (where the WebView it's rendered) is visible or not, if isn't visible the MainActivity will be shown and immediately will be hidden back.
And the app will be removed from the last used applications



Method inside my ForegroundServiceHelper Interface

public void InitBackgroundWebView() {
    if ((bool)SharedSettings.Entries.Get("MainPage.IsVisible") == false) {
        // Shows the activity
        Intent serviceIntent = new Intent(context, typeof(MainActivity));
        serviceIntent.AddFlags(ActivityFlags.NewTask);
        context.StartActivity(serviceIntent);
        // And immediately hides it back
        Intent main = new Intent(Intent.ActionMain);
        main.AddFlags(ActivityFlags.NewTask);
        main.AddCategory(Intent.CategoryHome);
        context.StartActivity(main);
        // Removes from the last app used
        ActivityManager am = (new ContextWrapper(Android.App.Application.Context)).GetSystemService(Context.ActivityService).JavaCast<ActivityManager>();
        if (am != null) {
            System.Collections.Generic.IList<ActivityManager.AppTask> tasks = am.AppTasks;
            if (tasks != null && tasks.Count > 0) {
                tasks[0].SetExcludeFromRecents(true);
            }
        }
    }
}


The SharedSettings class is an helper class wrapped around the App.Current.Properties Dictionary


And in the OnAppearing and OnDisappearing callbacks I set the shared values to true/false



[EDIT]
This workaround works only if the user is on the homepage, so I need to find an another solution...

user_97295490
Related