There has been a long-standing goal in some circles to allow Android devices to serve as desktop or notebook replacements, through the use of accessories. One of the first forays in this area was 2011’s Motorola Atrix, which offered a notebook-style docking station that allowed the Android device to be used with the dock’s keyboard and 11.5" display.

In recent years, Samsung has made a renewed push in this area, in the form of Samsung DeX.

The original 2017 DeX was a desktop docking station for the Galaxy S8/S8+ that provided power, an Ethernet jack, USB ports, and an HDMI port.

When docked, the Samsung phone would switch into a “DeX” mode that offers a freeform multiwindow experience. In this mode, the touchscreen is turned off, and the user navigates the windows using a keyboard and mouse (USB or Bluetooth), as with a traditional desktop OS.

In 2018, Samsung introduced the DeX Pad, which not only offers power, USB ports, and an HDMI port, but also allows the Samsung phone to serve as a touchpad for navigating the DeX environment:

The DeX Pad and original DeX dock now work with a variety of high-end Samsung devices, including:

On the whole, developers do not seem to be concerning themselves too much with DeX — for example, as of early August 2018, there were exactly two questions on Stack Overflow in the samsung-dex tag. That being said, the DeX is an interesting demonstration of Android’s freeform multi-window mode. Plus, it is yet another environment that puts keyboards and mice “front and center” for users and, by extension, app developers.

DeX Screen Modes

The user has two choices when docking their device in the DeX: screen mirroring mode and DeX mode.

Screen Mirroring

What Samsung describes as “screen mirroring” mode is pretty much what you would expect from an Android device connected to an HDMI display. By default, the contents of the touchscreen are mirrored on the HDMI display. And, if you use things like Presentation, you can display separate content on the HDMI display from what is shown on the touchscreen.

However, this mode may not be very popular, for one simple reason: the device is docked in the DeX in portrait mode. This means that the content shown on the HDMI device, by default, is in portrait mode. While you could lock your activity to landscape mode, so its Presentation appears in landscape, then the activity is in the wrong orientation on the touchscreen.

DeX Mode

More often than not, if people are bothering to put their devices in a DeX, it is to use Dex mode, with freeform-style windows.

This uses Android’s official freeform multi-window support. In theory, the experience should be somewhat reminiscent of how Android apps behave on Chrome OS, which also uses freeform multi-window.

There is an icon in the title bar that allows the user to rotate the window to landscape:

Activities that are resizeable — the <activity> or <application> explicitly has android:resizeableActivity="true" — can be resized by using a mouse and dragging the window edges, as with a traditional desktop operating system.

All windows, resizeable or not, can be minimized, putting them in an application dock at the bottom of the DeX screen.

Other App Impacts

While the screen, keyboard, and mouse are the primary changes that app developers will face when thinking about the DeX, there are other issues to take into account and opportunities to consider.

Configuration and State Changes

The strangest part of life in a DeX dock comes in the various state changes that your app can undergo. Sometimes, these are part of standard Android configuration changes. Sometimes, these are more distinctive to the DeX.

Moving In and Out of DeX

(in theory, there should also be “a docked device is switched from screen-mirroring mode to DeX mode”, but there does not appear to be an option for this)

Going into Dex mode will have your foreground app appear in the “system tray”, which serves the same role in desktop mode as the overview screen does in standard Android. Clicking your app’s icon will bring up your window… starting your app along the way.

The reason why Samsung does this by default is that switching to and from DeX mode is a fairly major change to your environment, specifically amounting to these configuration changes:

In addition, there is a fairly substantial resolution change (e.g., 2960x1440 to whatever the HDMI display supports).

If you can handle all of those changes without harming the user experience, add this <meta-data> element as a child of the <application> element in your manifest:


<meta-data android:name="com.samsung.android.keepalive.density" android:value="true"/>

With that element in place, when the user enters or exits DeX mode (e.g., puts the device into or removes it from the dock), your app is supposed to be left alone. In practice, this does not appear to work.

Resource Set Qualifiers

DeX honors the -desk UI mode resource set qualifier. If you want a different layout for a desktop-style UI compared to a mobile-style UI, you could have dedicated layout resources in res/layout-desk/ or related resource sets.

Networking

If the DeX dock is plugged into Ethernet, the device will switch to Ethernet connectivity when it is placed into the dock by default, though the user can configure this behavior.

If your app assumes that the Internet connection is via WiFi or mobile data, this is yet another scenario in which it might not be, to go along with Android TV, Android Things, and various miscellaneous Android devices offering Ethernet ports (e.g., Jide’s Remix Mini PC).

Also, this means that your connectivity may change when the device is put into or removed from the dock… but, as noted above, your app may be killed anyway.

USB

In addition to working with keyboards and mice, the USB ports on the DeX dock are standard USB hosts. So, for example, the user can plug in a flash drive and browse that drive’s contents.

In principle, the user can plug in any sort of USB device, and so long as the Samsung device has the appropriate drivers (or your app uses the USB APIs), the device should work.

Note that the USB ports are USB 2.0, not 3.0 or higher. Also note that they are classic Type A connectors, even though the power supply input comes from a Type C connector.

Debugging Interface

The DeX dock requires that you power it through a high-power USB port. A standard 500mA USB port will be insufficient. The problem is that most high-power USB ports do not also support data, and in those situations you cannot use the USB cable for debugging.

Fortunately, what is often called “WiFi debugging” works, though technically it may be occurring over Ethernet, if the DeX dock has an Ethernet cable plugged in and the device is using it.

You should get a message showing that you are connected, adb devices should show the connection, and Android Studio should be able to work with the Samsung device.

Use adb disconnect ..., where ... is the IP address of the device, to drop this debugging connection.

Screenshots

When the docked device is in DeX mode, you cannot take screenshots through normal means (e.g., Android Studio, adb shell screencap). As with Android Things, you can record screencasts. And, given a short screencast, you can extract a frame to use as a screenshot, such as via the command-line ffmpeg utility.

However, the screencast (and any resulting screenshot) may be off, depending on the screen resolution of whatever HDMI display you have plugged the DeX dock into. The screencast will record at 2960x1440 resolution, which is the resolution of the touchscreen. However, it is very likely that your HDMI display cannot show that, and so the DeX dock will display at something like 1080p (1920x1080). The screencast will be at 2960x1440 resolution, with the external display’s content shown in the upper-left corner, and black pixels filling the rest of the 2960x1440 frame.

For screenshots, you can crop the raw screenshot to get the portion that you need, dropping out the extraneous black bands:


adb shell screenrecord --verbose /sdcard/screen.mp4 --time-limit 1
adb pull /sdcard/screen.mp4
adb shell rm /sdcard/screen.mp4
ffmpeg -ss 0 -i screen.mp4 -t 1 -s 2960x1440 -f image2 temp.png
convert temp.png -crop 1920x1080+0+0 +repage result.png

Here, we use adb, ffmpeg, and ImageMagick’s convert tools to record a one-second screencast, clip a 2960x1440 frame from it (to temp.png), then crop the upper-left 1920x1080 pixels to form the final screenshot (result.png).

For screencasts, there should be an equivalent way of having ffmpeg crop all of the frames to the desired resolution. The proof of this is left as an exercise for the reader.

Detecting DeX

Ideally, your app does not care whether it is running on a device in a DeX dock or not. In theory, if you do everything correctly in general (particularly multi-window support), then “it just works”.

If you run into some DeX-specific quirk that you need to work around, you cannot use traditional Build values like Build.PRODUCT, because those reflect the device, not the dock.

Instead, Samsung apparently extended the android.content.res.Configuration class to have a DeX-specific field on DeX-capable devices:


private boolean iCanHazDeX() {
  boolean result=false;
  Configuration config=getResources().getConfiguration();

  try {
    Class clsConfig=config.getClass();

    if (clsConfig.getField("SEM_DESKTOP_MODE_ENABLED").getInt(clsConfig)
      ==clsConfig.getField("semDesktopModeEnabled").getInt(config)) {
      result=true;
    }
  }
  catch(Exception e) {
    // guess not!
  }

  return(result);
}

Drag-and-Drop

The Samsung DeX environment supports cross-window drag-and-drop within one app. It also supports cross-app drag-and-drop, so one app can advertise as a drop target and other apps can allow drag-and-drop to the first app.

For More Information

Samsung has several pages devoted to developer documentation for the DeX, including:

Device Catalog: Samsung DeX