HomeCreate a UWP document scanning app

Create a UWP document scanning app

The Universal Windows Platform (UWP) lets us create and publish apps for any Windows device. This is Microsoft’s latest technology for creating client apps. We can use the latest visuals and features of the latest version of Windows in the UWP app.

Dynamic Web Twin (DWT) is a JavaScript document scanning library. If we want to use it to create a document scanning app with UWP, we can use UWP’s WebView control. A web view control embeds a view into your app that presents web content using the Microsoft Edge Legacy rendering engine.

This solution is a hybrid app. It uses local UWP controls as well as a web page.

In this article, we are going to create UWP document scanning app in steps.

An overview of how to implement document scanning with UWP.

  1. Create a new Universal Windows App Project
  2. Copy the Web Twin Resource folder and demo into the project.
  3. Create a webview control and load the HTML file.
  4. Override the default behavior when the web detects that TWAIN is not installed.
  5. Use the capabilities provided by the Windows Runtime API.

needs.

  1. Visual Studio with Windows 10 SDK.
  2. Dynamic Web TWAIN

Create a UWP document scanning app

New Universal Windows App Project

Open Visual Studio and create a new Universal Windows app.

Copy the Web Twin Resources folder and demo into the project.

  1. Create a new folder named DWT.
  2. Download the pre-built demo (link) HTML, CSS and JavaScript files and keep them below. DWT Folder
  3. Copy Resources Folder from C:Program Files (x86)DynamsoftDynamic Web TWAIN SDK 17.1 To do DWT Folder. The path may change depending on your environment.

Paste the resource folder into the DWT folder.

Create a webview control and load the HTML file.

  1. Edit MainPage.xaml File with the following code:

    <RelativePanel>
         <WebView x:Name="WebView1" RelativePanel.AlignTopWithPanel="True" RelativePanel.AlignLeftWithPanel="True" RelativePanel.AlignRightWithPanel="True" RelativePanel.AlignBottomWithPanel="True" LoadCompleted="WebView1_LoadCompleted"/>
     </RelativePanel>
  2. Edit MainPage.xaml.cs File to navigate WebView to HTML file when the app starts:

     public MainPage()
     {
         this.InitializeComponent();
         WebView1.Navigate(new Uri("ms-appx-web:///DWT/index.html"));
     }

If you have Dynamic Web TWAIN installed, you should be able to scan documents within WebView.

Dynamic Web Twin: Scan documents within WebView.

If you have not done so, you may encounter this dialogue.

Dialog: Please complete a one-time setup.

PS: Dynamic Web TWAIN’s JavaScript library needs to communicate with a local service to provide a document scanning function. Users must first install the service to use the scanning app.

Override the default behavior when the web detects that TWAIN is not installed

When Web TWAIN is not detected, it will show the Download dialog by default. But it is not possible to download the installer inside Webview. We can edit the default behavior and open the download link using the system’s browser.

UWP’s WebView Class provides a way to notify a request from a web page.

Use window.external.notify From the HTML event handler to notify the application using. WebView.ScriptNotify.

In the C # file, we add the following code to enable the page to notify the app.

public MainPage()
{
    this.InitializeComponent();
    List<Uri> allowedUris = new List<Uri>();  //newly added
    allowedUris.Add(new Uri("ms-appx-web:///DWT/index.html"));  //newly added
    WebView1.ScriptNotify += WebView1_ScriptNotify;  //newly added
    WebView1.Navigate(new Uri("ms-appx-web:///DWT/index.html"));
}

//newly added
private async void WebView1_ScriptNotify(object sender, NotifyEventArgs e) 
{
    //handles the message
}

Open up dynamsoft.webtwain.install.js File in Resources Folder. Change the following code:

Dynamsoft.OnWebTwainNotFoundOnWindowsCallback = function (ProductName, InstallerUrl, bHTML5, bIE, bSafari, bSSL, strIEVersion) {
    var _this = Dynamsoft, objUrl = { 'default': InstallerUrl };
    _this._show_install_dialog(ProductName, objUrl, bHTML5, Dynamsoft.DWT.EnumDWT_PlatformType.enumWindow, bIE, bSafari, bSSL, strIEVersion);
};

For him:

Dynamsoft.OnWebTwainNotFoundOnWindowsCallback = function (ProductName, InstallerUrl, bHTML5, bIE, bSafari, bSSL, strIEVersion) {
    var response = {};
    response["info"] = "dynamsoft_service_not_running";
    window.external.notify(JSON.stringify(response));
};

The web page will notify the app when it finds out that Web TWAIN is not installed. This will give the app the JSON string.

In the C # file, we use the following code to download and install the service from the user.

private async void WebView1_ScriptNotify(object sender, NotifyEventArgs e)
{
    var response = JsonConvert.DeserializeObject<Dictionary<string, string>>(e.Value);
    string info = response.GetValueOrDefault("info", "");
    // Respond to the script notification.
    if (info == "dynamsoft_service_not_running")
    {
        // Create the message dialog and set its content
        var messageDialog = new MessageDialog("Dynamsoft Service is not running. Please download and install it.");

        // Add commands and set their callbacks; both buttons use the same callback function instead of inline event handlers
        messageDialog.Commands.Add(new UICommand(
            "Download",
            new UICommandInvokedHandler(this.CommandInvokedHandler)));
        messageDialog.Commands.Add(new UICommand(
            "Close",
            new UICommandInvokedHandler(this.CommandInvokedHandler)));

        // Set the command that will be invoked by default
        messageDialog.DefaultCommandIndex = 0;

        // Set the command to be invoked when escape is pressed
        messageDialog.CancelCommandIndex = 1;

        // Show the message dialog
        await messageDialog.ShowAsync();
    }
}

private async void CommandInvokedHandler(IUICommand command)
{
    if (command.Label == "Download") {
        string uriToLaunch = @"https://download.dynamsoft.com/Demo/DWT/DWTResources/dist/DynamsoftServiceSetup.msi";
        var uri = new Uri(uriToLaunch);
        var success = await Windows.System.Launcher.LaunchUriAsync(uri);
    }
}

Dialog that shows whether the service is running:

Conversation: Dynamsoft service is not running.

Use the capabilities provided by the Windows Runtime API.

As it is in a WebView environment, some functions of Web TWAIN are not available, such as camera add-on (webcam add-on works because it uses a local service). Since this is a UWP application, we can use UWP’s Windows Runtime API to enhance its functions.

Use the Camera API.

Windows.Media.Capture.CameraCaptureUI API provides easy-to-use camera interface.

private async void CameraButton_Click(object sender, RoutedEventArgs e)
{
    // Using Windows.Media.Capture.CameraCaptureUI API to capture a photo
    CameraCaptureUI dialog = new CameraCaptureUI();
    dialog.VideoSettings.AllowTrimming = true;
    StorageFile file = await dialog.CaptureFileAsync(CameraCaptureUIMode.Photo);
}

Camera interface.

A built-in camera dialog will appear for taking pictures. It also supports automatic cropping for document scanning.

Once the image is captured, we can send it to the web page. WebView.InvokeScriptAsync Method

In C # file:

private async void CameraButton_Click(object sender, RoutedEventArgs e)
{
    // Using Windows.Media.Capture.CameraCaptureUI API to capture a photo
    CameraCaptureUI dialog = new CameraCaptureUI();
    dialog.VideoSettings.AllowTrimming = true;
    StorageFile file = await dialog.CaptureFileAsync(CameraCaptureUIMode.Photo);
    string base64 = await StorageFileToBase64(file);  //newly added
    await WebView1.InvokeScriptAsync("LoadImageFromBase64", new string[] { base64 });  //newly added
}

//https://stackoverflow.com/questions/18553691/metro-getting-the-base64-string-of-a-storagefile
private async Task<string> StorageFileToBase64(StorageFile file)
{
    string Base64String = "";

    if (file != null)
    {
        IRandomAccessStream fileStream = await file.OpenAsync(FileAccessMode.Read);
        var reader = new DataReader(fileStream.GetInputStreamAt(0));
        await reader.LoadAsync((uint)fileStream.Size);
        byte[] byteArray = new byte[fileStream.Size];
        reader.ReadBytes(byteArray);
        Base64String = Convert.ToBase64String(byteArray);
    }

    return Base64String;
}

In the HTML file, get the base64 encoded image:

function LoadImageFromBase64(base64) {
    if (DWObject) {
        DWObject.LoadImageFromBase64Binary(
            base64,
            Dynamsoft.DWT.EnumDWT_ImageType.IT_ALL);
    }
}

Use the OCR API.

The Windows.Media.Ocr API provides an easy-to-use OCR interface.

Here, we add a function to the OCR selected image using this API.

In the HTML file, add a function to get the selected image base 64:

function GetSelectedImageInBase64() {
    if (DWObject) {
        DWObject.ConvertToBase64(
            [DWObject.CurrentImageIndexInBuffer],
            Dynamsoft.DWT.EnumDWT_ImageType.IT_PNG,
            function (result, indices, type) {
                var data = result.getData(0, result.getLength());
                var response = {};
                response["info"] = "image_base64";
                response["data"] = data;
                window.external.notify(JSON.stringify(response));
            },
            function (errorCode, errorString) {
                console.log(errorString);
            }
        );
    }
}

In the C # file, call the above method, retrieve the base 64 encoded image and OCR it:

private async void OCRButton_Click(object sender, RoutedEventArgs e)
{
    await WebView1.InvokeScriptAsync("GetSelectedImageInBase64", new string[] { });
}

private async void WebView1_ScriptNotify(object sender, NotifyEventArgs e)
{
    if (info == "image_base64") {
        if (response.ContainsKey("data")) {
            string base64 = response.GetValueOrDefault("data","");
            OCRImageFromBase64(base64);
        }
    }
}

private async void OCRImageFromBase64(string base64) {
    byte[] bytes;
    bytes = Convert.FromBase64String(base64);
    IBuffer buffer = WindowsRuntimeBufferExtensions.AsBuffer(bytes, 0, bytes.Length);
    InMemoryRandomAccessStream inStream = new InMemoryRandomAccessStream();
    DataWriter datawriter = new DataWriter(inStream.GetOutputStreamAt(0));
    datawriter.WriteBuffer(buffer, 0, buffer.Length);
    await datawriter.StoreAsync();
    BitmapDecoder decoder = await BitmapDecoder.CreateAsync(inStream);
    SoftwareBitmap bitmap = await decoder.GetSoftwareBitmapAsync(BitmapPixelFormat.Bgra8, BitmapAlphaMode.Premultiplied);
    OcrEngine ocrEngine = OcrEngine.TryCreateFromUserProfileLanguages();
    OcrResult ocrResult = await ocrEngine.RecognizeAsync(bitmap);
    ContentDialog contentDialog = new ContentDialog
    {
        Title = "Result:",
        Content = ocrResult.Text,
        PrimaryButtonText = "Copy to clipboard",
        CloseButtonText = "Close"
    };

    ContentDialogResult result = await contentDialog.ShowAsync();

    if (result == ContentDialogResult.Primary)
    {
        DataPackage dataPackage = new DataPackage();
        dataPackage.SetText(ocrResult.Text);
        Clipboard.SetContent(dataPackage);
    }
}

A content dialog will appear with the OCR result. Users can copy text to the clipboard.

OCR result

Source code

Download the project and try it yourself: https://github.com/xulihang/UWP-Document-Scanner

.