Thursday, February 16, 2012

Handle shortcuts with GetKeyState

Recently, I found myself in a situation where some 3rd party code was blocking certain shortcuts from tunneling down to my form. The shortcuts were intercepted before the Form's OnKeyDown event, and the PreviewKeyDown event. To my dismay, they were a no-show in ProcessCmdKey as well.

Luckily, we have platinum support for the vendor in question, so I contacted support and asked them what exactly is intercepting these shortcuts and how to circumvent it. It seems no one could give me a straight answer, but they did offer to sell me a "source code license" for the low low price of $xxxx. I spent some time going through the scores of obfuscated classes my decompiler spat out with no luck, but if I have to guess the shortcuts in question were probably intercepted by an IMessageFilter hooked up to our application.

Therefore, the .Net solution was out, WinAPI it is! I spent hours looking at half-working, half-assed solutions involving GetMessage / PeekMessage and a few involving GetAsyncKeyState. Now, when I say half-working I mean most of the time the solution didn't take modifier keys into account, was completely detached from .Net-land or was outright broken. When I say half-assed, I mean hard-coded case statements of 3 key shortcuts right there in the class, checking that the result value of GetAsyncKeyState is non-zero and calling it a day.

Needless to say, I ended up rolling my own solution. After some performance profiling, and after taking a quick gander at GetKeyState vs. GetAsyncKeyState I went with a solution that uses GetKeyState.


The class is a singleton, partly for ease of use, but mostly because I had to get it into some places where dependency injection just wasn't an option (don't ask). You register a shortcut to an object (usually a form), listen to an event that fires when a shortcut is triggered and unregister the shortcut when the subscribed object is disposed. Easy, right?

Register shortcuts

'When (un)registering "a lot" of shortcuts or subscribers, it's faster to just pause capture while doing it
GlobalShortcutObserver.GetInstance.RegisterShortcut(Me, Shortcut.CtrlW)
GlobalShortcutObserver.GetInstance.RegisterShortcut(Me, Shortcut.CtrlA)
GlobalShortcutObserver.GetInstance.RegisterShortcut(Me, Shortcut.CtrlS)
GlobalShortcutObserver.GetInstance.RegisterShortcut(Me, Shortcut.CtrlD)
GlobalShortcutObserver.GetInstance.RegisterShortcut(Me, Shortcut.CtrlShiftS)

Unregister shortcuts


Shortcut event handler

'Executes when any globally registered shortcut is triggered
Private Sub Event_GlobalShortcutObserver_OnShortcutPressed(Subscriber As Object, Shortcut As Shortcut)
 'This shortcut was not intended for us
 If Subscriber IsNot Me Then Return

 MsgBox("You pressed " & Shortcut.ToString())
End Sub

The goods

Imports System.Windows.Forms

Public Class GlobalShortcutObserver
 Implements IDisposable

 Public Declare Function GetKeyState Lib "user32" (ByVal vKey As Integer) As Short

 'Holds a list of all subscribers for a given shortcut
 Private mShortcutToSubscriber As Dictionary(Of Shortcut, List(Of Object))

 'Breaks down a shortcut to its modifier and key components for faster checks
 Private mModifiersToKey As Dictionary(Of Modifier, List(Of Keys))

 'A timer that polls the keystate of registered shortcuts
 Private mKeyboardPollHeartBeat As Timer
 Const KeyboardPollHeartBeatInterval As Integer = 25 'How often the heartbeat ticks
 Const KeyboardPollHeartBeatDelay As Integer = 250 'How much we delay the next tick after a valid shortcut is pressed

 Const VK_MENU As Integer = &H12
 Const VK_SHIFT As Integer = &H10
 Const VK_CONTROL As Integer = &H11
 Const SIGNIFICANT_BIT As Integer = &H8000

 Private Enum Modifier
  None = 0
  Alt = 1
  Ctrl = 1 << 1
  Shift = 1 << 2
 End Enum

#Region "Public"
 Public Event OnShortcutPressed(Subscriber As Object, Shortcut As Shortcut)

 Public Shared Function GetInstance() As GlobalShortcutObserver
  Static Instance As New GlobalShortcutObserver
  Return Instance
 End Function

 Private Sub New()
  Me.mShortcutToSubscriber = New Dictionary(Of Shortcut, List(Of Object))
  Me.mModifiersToKey = New Dictionary(Of Modifier, List(Of Keys))

  'Periodically poll the keyboard if a shotcut is pressed
  Me.mKeyboardPollHeartBeat = New Timer() With {.Interval = KeyboardPollHeartBeatInterval}
 End Sub

 ''' Pausees shortcut capturing.
 ''' Used when registering/unregistering a large amount of shortcuts
 Public Sub PauseCapture()
 End Sub

 ''' Resumes shortcut capturing.
 ''' Used when registering/unregistering a large amount of shortcuts
 Public Sub ResumeCapture()
 End Sub

 ''' Registers a single shortcut for a a given subscriber.
 Public Sub RegisterShortcut(Subscriber As Object, Shortcut As Shortcut)
  Dim Composite = Me.GetShortcutComposite(Shortcut)

  'Populate the ModifierToKey collection last, since this is used to probe for existing shortcuts by the polling timer.
  If Not Me.mShortcutToSubscriber.ContainsKey(Shortcut) Then
   Me.mShortcutToSubscriber.Add(Shortcut, New List(Of Object))
  End If

  If Not Me.mShortcutToSubscriber(Shortcut).Contains(Subscriber) Then
  End If

  If Not Me.mModifiersToKey.ContainsKey(Composite.Modifiers) Then
   Me.mModifiersToKey.Add(Composite.Modifiers, New List(Of Keys))
  End If

  If Not Me.mModifiersToKey(Composite.Modifiers).Contains(Composite.Key) Then
  End If
 End Sub

 ''' Unregisters all shortcuts for a given subscriber.
 Public Sub UnregisterShortcuts(Subscriber As Object)
  Dim Composite As ShortcutComposite

  'Remove the subscriber
  For Each Subscribers In Me.mShortcutToSubscriber.Values
   If Subscribers.Contains(Subscriber) Then
   End If

  'Remove any shortcut that has no more subscribers
  For Each Row In Me.mShortcutToSubscriber.Where(Function(x) x.Value.Count = 0).ToList

   'Remove the current Key for this Modifier
   Composite = Me.GetShortcutComposite(Row.Key)

   'If that Modifier has no more Keys associate, we should remove it as well
   If Me.mModifiersToKey(Composite.Modifiers).Count = 0 Then
   End If
 End Sub
#End Region

#Region "Private"
 Private Structure ShortcutComposite
  Public Modifiers As Modifier
  Public Key As Keys

  Public Sub New(Modifiers As Modifier, Key As Keys)
   Me.Modifiers = Modifiers
   Me.Key = Key
  End Sub
 End Structure

 'Breaks down a Shortcut into a Keys enum and a Modifiers bitmask
 Private Function GetShortcutComposite(Shortcut As Shortcut) As ShortcutComposite
  Dim strShortcut = Shortcut.ToString
  Dim Key As Keys = Nothing
  Dim Modifiers = Modifier.None
  Dim CurrentModifier As Modifier

  'Strip out the modifiers from the passed in shortcut and append them to our Modifiers bitmask
  'Note: Since we do all our string checks as StartsWith, we have to get the modifiers list in the correct order
  For Each strModifier In New String() {"Alt", "Ctrl", "Shift"}
   If strShortcut.StartsWith(strModifier) Then
    'Convert our string modifier into a valid KeyModifier
    If Not [Enum].TryParse(strModifier, CurrentModifier) Then
     Throw New ApplicationException(String.Format("Could not recognize modifier ""{0}"".", strModifier))
    End If

    'Add the current modifier to our modifiers bitmask
    Modifiers = Modifiers Or CurrentModifier

    'And strip out the modifier from the shortcut string
    strShortcut = strShortcut.Replace(strModifier, "")
   End If

  'At this point we should have a single key in the shortcut string a valid modifiers bitmask.
  'Convert our string into a valid Keys enum.
  If Not [Enum].TryParse(strShortcut, Key) Then
   Throw New ApplicationException(String.Format("Could not recognize Key ""{0}"".", strShortcut))
  End If

  Return New ShortcutComposite(Modifiers, Key)
 End Function

 'Checks if a known shortcut is pressed and notifies all subscribers if it is
 Private Sub Event_KeyboardPollHeartBeat_Tick(sender As Object, e As EventArgs)
   If Me.mKeyboardPollHeartBeat.Interval <> KeyboardPollHeartBeatInterval Then
    Me.mKeyboardPollHeartBeat.Interval = KeyboardPollHeartBeatInterval
   End If

   Dim Modifiers = Modifier.None
   Dim strModifiers = ""
   Dim KeysPressed As List(Of Keys)
   Dim CurrentShortcut As Shortcut
   Dim strCurrentShortcut As String

   If (GetKeyState(VK_MENU) And SIGNIFICANT_BIT) <> 0 Then
    Modifiers = Modifiers Or Modifier.Alt
    strModifiers &= Modifier.Alt.ToString
   End If

   If (GetKeyState(VK_CONTROL) And SIGNIFICANT_BIT) <> 0 Then
    Modifiers = Modifiers Or Modifier.Ctrl
    strModifiers &= Modifier.Ctrl.ToString
   End If

   If (GetKeyState(VK_SHIFT) And SIGNIFICANT_BIT) <> 0 Then
    Modifiers = Modifiers Or Modifier.Shift
    strModifiers &= Modifier.Shift.ToString
   End If

   'No modifiers have been pressed, this can't be a shortcut
   If Modifiers = Modifier.None Then Return

   'No shortcut is associated with the current modifiers
   If Not Me.mModifiersToKey.ContainsKey(Modifiers) Then Return

   'Determine if any of the keys associated with this modifier are pressed
   KeysPressed = New List(Of Keys)
   For Each Key In Me.mModifiersToKey(Modifiers).ToList 'Guard against mid-loop mutation
    If (GetKeyState(Key) And SIGNIFICANT_BIT) <> 0 Then
    End If

   'No keys that were expected with the current modifier are pressed
   If KeysPressed.Count = 0 Then Return

   'Let's put a bit of delay before we capture the next shortcut
   Me.mKeyboardPollHeartBeat.Interval = KeyboardPollHeartBeatDelay

   'Time to reconsitute the modifiers and keys into a shortcut and notify the appropriate subscribers
   For Each Key In KeysPressed
    strCurrentShortcut = strModifiers & Key.ToString

    'Convert our string into a valid Shortcut enum.
    If Not [Enum].TryParse(strCurrentShortcut, CurrentShortcut) Then
     Throw New ApplicationException(String.Format("Could not recognize Shortcut ""{0}"".", strCurrentShortcut))
    End If

    'Notify all subscribers to this shortcut
    For Each Subscriber In Me.mShortcutToSubscriber(CurrentShortcut).ToList 'Guard against mid-loop mutation
     RaiseEvent OnShortcutPressed(Subscriber, CurrentShortcut)

  Catch ex As Exception
  End Try
 End Sub

 Private Sub AttachKeyboardPollHeartBeatEvents(Optional ByVal RemoveOnly As Boolean = False)
  RemoveHandler Me.mKeyboardPollHeartBeat.Tick, AddressOf Me.Event_KeyboardPollHeartBeat_Tick
  If Not RemoveOnly Then AddHandler Me.mKeyboardPollHeartBeat.Tick, AddressOf Me.Event_KeyboardPollHeartBeat_Tick
 End Sub
#End Region

#Region "IDisposable"
 Private mDisposedValue As Boolean = False

 Public Sub Dispose() Implements IDisposable.Dispose
 End Sub

 Protected Overridable Sub Dispose(ByVal Disposing As Boolean)
  If Not Me.mDisposedValue Then
   If Disposing Then
    'Get rid of the heartbeat object
   End If
  End If

  Me.mDisposedValue = True
 End Sub
#End Region
End Class