jprdonnelly/terminal
1
0
Fork 0
mirror of https://github.com/microsoft/terminal.git synced 2025-12-19 18:11:39 -05:00
Code Issues Projects Releases Wiki Activity

[wpf] Add UIA events (#14097)

Browse Source 
Adds UIA events to the WPF control for the following items:
- selection changed
- text changed (and output)
- cursor changed

### Automation Peer
Similar to the architecture of the UWP TermControl, we added a
`HwndTerminalAutomationPeer` which acts as the
`TermControlAutomationPeer` in UWP. However, we don't need a XAML
wrapper here, so really we just need it to inherit from
`TermControlUiaProvider` (the `ITextProvider` implementation shared
across conhost and WT) and `IUiaEventDispatcher` (the event dispatching
interface that is responsible for signaling the screen reader that
something has changed).

### Removing the local echo
As with WT, we need to record key events to remove the local echo. These
recorded events are matched up with the output text. Each sequential
match is removed in the output text so that it's not read by the screen
reader.

### Detecting what to send events for
As with WT, a `UiaEngine` was added to the renderer and it is set up
when a UIA client is detected. WT would normally stop sending events
when focus was lost from the control. We do the same here.

### Automation properties
`TermControlUiaProvider` was upgraded to support property values. Such
properties include class name and control type. These align with those
set in `TermControlAutomationPeer`. Realistically, those should point to
these, but that requires a lot more work and a localization burden
(because we need to move the localized word "terminal").

`HwndTerminalAutomationPeer` takes this a step further and overrides the
class name to be `WPFTermControl`. This allows screen readers to provide
special handling for the `WPFTermControl` vs the UWP term control since
they will be updating at different speeds.

### Build fixes
To build the WPF test app, I had to mess with the dependencies a little
bit. Really just add the atlas engine and uia renderer to the build
steps.

### HwndTerminal initialization
The initialization order with `WM_NCCREATE` was changed to match that of
Windows Terminal (BaseWindow/IslandWindow). This is safer now. I also
removed the `static` window because it was unnecessary.

### Handling `WM_GETOBJECT`
WPF's HwndHost likes to mark the `WM_GETOBJECT` message as handled to
force the usage of the WPF automation peer. We now explicitly mark it as
not handled and don't return an automation peer. This forces the message
to go down to the HwndTerminal where we return terminal's UiaProvider.

### Remove TermControl layer from UIA tree
TermContol (the top-most layer in the UIA tree) would pop up and not do
anything. This PR also overrides the automation peer at that layer and
marks IsContentElement/IsControlElement=false (the equivalent to
AccessibilityView=Raw). This makes the layer only appear in the UIA tree
if you are using the raw view (i.e. you know what you're doing and you
want to see each individual layer even if you can't directly interact
with it).

## Validation Steps Performed
Tested with Narrator/NVDA using WpfTerminalTestNetCore project in our
repo.
- [X] New output is read out (not just key events, but also other output
  text)
- [X] Local echo does not occur (i.e. pressing 'A' should only read 'A'
  once, not twice [key event and rendered letter]).
- [X] selection events are read out properly
- [X] cursor change events are read out properly (tested with text
  cursor indicator preview in Settings App > Accessibility > Text
  Cursor)

NOTE: test this with Release builds. Debug builds may be too slow and
not read out properly

Closes #12642
This commit is contained in:
Carlos Zamora
2022-10-06 16:11:47 -07:00
committed by GitHub
parent 40bc3d7fbc
commit 5608cf15a3
12 changed files with 370 additions and 45 deletions
Download Patch File Download Diff File Expand all files Collapse all files

18
OpenConsole.sln
View File

@@ -643,7 +643,8 @@ Global
{DCF55140-EF6A-4736-A403-957E4F7430BB}.Debug|ARM.ActiveCfg = Debug|Win32
{DCF55140-EF6A-4736-A403-957E4F7430BB}.Debug|ARM64.ActiveCfg = Debug|ARM64
{DCF55140-EF6A-4736-A403-957E4F7430BB}.Debug|ARM64.Build.0 = Debug|ARM64
{DCF55140-EF6A-4736-A403-957E4F7430BB}.Debug|DotNet_x64Test.ActiveCfg = Debug|Win32
{DCF55140-EF6A-4736-A403-957E4F7430BB}.Debug|DotNet_x64Test.ActiveCfg = Debug|x64
{DCF55140-EF6A-4736-A403-957E4F7430BB}.Debug|DotNet_x64Test.Build.0 = Debug|x64
{DCF55140-EF6A-4736-A403-957E4F7430BB}.Debug|DotNet_x86Test.ActiveCfg = Debug|Win32
{DCF55140-EF6A-4736-A403-957E4F7430BB}.Debug|x64.ActiveCfg = Debug|x64
{DCF55140-EF6A-4736-A403-957E4F7430BB}.Debug|x64.Build.0 = Debug|x64
@@ -661,7 +662,8 @@ Global
{DCF55140-EF6A-4736-A403-957E4F7430BB}.Release|ARM.ActiveCfg = Release|Win32
{DCF55140-EF6A-4736-A403-957E4F7430BB}.Release|ARM64.ActiveCfg = Release|ARM64
{DCF55140-EF6A-4736-A403-957E4F7430BB}.Release|ARM64.Build.0 = Release|ARM64
{DCF55140-EF6A-4736-A403-957E4F7430BB}.Release|DotNet_x64Test.ActiveCfg = Release|Win32
{DCF55140-EF6A-4736-A403-957E4F7430BB}.Release|DotNet_x64Test.ActiveCfg = Release|x64
{DCF55140-EF6A-4736-A403-957E4F7430BB}.Release|DotNet_x64Test.Build.0 = Release|x64
{DCF55140-EF6A-4736-A403-957E4F7430BB}.Release|DotNet_x86Test.ActiveCfg = Release|Win32
{DCF55140-EF6A-4736-A403-957E4F7430BB}.Release|x64.ActiveCfg = Release|x64
{DCF55140-EF6A-4736-A403-957E4F7430BB}.Release|x64.Build.0 = Release|x64
@@ -2308,7 +2310,8 @@ Global
{48D21369-3D7B-4431-9967-24E81292CF63}.Debug|ARM.ActiveCfg = Debug|Win32
{48D21369-3D7B-4431-9967-24E81292CF63}.Debug|ARM64.ActiveCfg = Debug|ARM64
{48D21369-3D7B-4431-9967-24E81292CF63}.Debug|ARM64.Build.0 = Debug|ARM64
{48D21369-3D7B-4431-9967-24E81292CF63}.Debug|DotNet_x64Test.ActiveCfg = Debug|Win32
{48D21369-3D7B-4431-9967-24E81292CF63}.Debug|DotNet_x64Test.ActiveCfg = Debug|x64
{48D21369-3D7B-4431-9967-24E81292CF63}.Debug|DotNet_x64Test.Build.0 = Debug|x64
{48D21369-3D7B-4431-9967-24E81292CF63}.Debug|DotNet_x86Test.ActiveCfg = Debug|Win32
{48D21369-3D7B-4431-9967-24E81292CF63}.Debug|x64.ActiveCfg = Debug|x64
{48D21369-3D7B-4431-9967-24E81292CF63}.Debug|x64.Build.0 = Debug|x64
@@ -2325,7 +2328,8 @@ Global
{48D21369-3D7B-4431-9967-24E81292CF63}.Release|ARM.ActiveCfg = Release|Win32
{48D21369-3D7B-4431-9967-24E81292CF63}.Release|ARM64.ActiveCfg = Release|ARM64
{48D21369-3D7B-4431-9967-24E81292CF63}.Release|ARM64.Build.0 = Release|ARM64
{48D21369-3D7B-4431-9967-24E81292CF63}.Release|DotNet_x64Test.ActiveCfg = Release|Win32
{48D21369-3D7B-4431-9967-24E81292CF63}.Release|DotNet_x64Test.ActiveCfg = Release|x64
{48D21369-3D7B-4431-9967-24E81292CF63}.Release|DotNet_x64Test.Build.0 = Release|x64
{48D21369-3D7B-4431-9967-24E81292CF63}.Release|DotNet_x86Test.ActiveCfg = Release|Win32
{48D21369-3D7B-4431-9967-24E81292CF63}.Release|x64.ActiveCfg = Release|x64
{48D21369-3D7B-4431-9967-24E81292CF63}.Release|x64.Build.0 = Release|x64
@@ -3313,7 +3317,8 @@ Global
{8222900C-8B6C-452A-91AC-BE95DB04B95F}.Debug|ARM.ActiveCfg = Debug|Win32
{8222900C-8B6C-452A-91AC-BE95DB04B95F}.Debug|ARM64.ActiveCfg = Debug|ARM64
{8222900C-8B6C-452A-91AC-BE95DB04B95F}.Debug|ARM64.Build.0 = Debug|ARM64
{8222900C-8B6C-452A-91AC-BE95DB04B95F}.Debug|DotNet_x64Test.ActiveCfg = Debug|Win32
{8222900C-8B6C-452A-91AC-BE95DB04B95F}.Debug|DotNet_x64Test.ActiveCfg = Debug|x64
{8222900C-8B6C-452A-91AC-BE95DB04B95F}.Debug|DotNet_x64Test.Build.0 = Debug|x64
{8222900C-8B6C-452A-91AC-BE95DB04B95F}.Debug|DotNet_x86Test.ActiveCfg = Debug|Win32
{8222900C-8B6C-452A-91AC-BE95DB04B95F}.Debug|x64.ActiveCfg = Debug|x64
{8222900C-8B6C-452A-91AC-BE95DB04B95F}.Debug|x64.Build.0 = Debug|x64
@@ -3333,7 +3338,8 @@ Global
{8222900C-8B6C-452A-91AC-BE95DB04B95F}.Release|ARM.ActiveCfg = Release|Win32
{8222900C-8B6C-452A-91AC-BE95DB04B95F}.Release|ARM64.ActiveCfg = Release|ARM64
{8222900C-8B6C-452A-91AC-BE95DB04B95F}.Release|ARM64.Build.0 = Release|ARM64
{8222900C-8B6C-452A-91AC-BE95DB04B95F}.Release|DotNet_x64Test.ActiveCfg = Release|Win32
{8222900C-8B6C-452A-91AC-BE95DB04B95F}.Release|DotNet_x64Test.ActiveCfg = Release|x64
{8222900C-8B6C-452A-91AC-BE95DB04B95F}.Release|DotNet_x64Test.Build.0 = Release|x64
{8222900C-8B6C-452A-91AC-BE95DB04B95F}.Release|DotNet_x86Test.ActiveCfg = Release|Win32
{8222900C-8B6C-452A-91AC-BE95DB04B95F}.Release|x64.ActiveCfg = Release|x64
{8222900C-8B6C-452A-91AC-BE95DB04B95F}.Release|x64.Build.0 = Release|x64

66
src/cascadia/PublicTerminalCore/HwndTerminal.cpp
View File

@@ -4,11 +4,7 @@
#include "pch.h"
#include "HwndTerminal.hpp"
#include <windowsx.h>
#include "../../types/TermControlUiaProvider.hpp"
#include <DefaultSettings.h>
#include "../../renderer/base/Renderer.hpp"
#include "../../renderer/dx/DxRenderer.hpp"
#include "../../cascadia/TerminalCore/Terminal.hpp"
#include "../../types/viewport.cpp"
#include "../../types/inc/GlyphWidth.hpp"
@@ -54,6 +50,17 @@ LRESULT CALLBACK HwndTerminal::HwndTerminalWndProc(
LPARAM lParam) noexcept
try
{
if (WM_NCCREATE == uMsg)
{
#pragma warning(suppress : 26490) // Win32 APIs can only store void*, have to use reinterpret_cast
auto cs = reinterpret_cast<CREATESTRUCT*>(lParam);
HwndTerminal* that = static_cast<HwndTerminal*>(cs->lpCreateParams);
that->_hwnd = wil::unique_hwnd(hwnd);
#pragma warning(suppress : 26490) // Win32 APIs can only store void*, have to use reinterpret_cast
SetWindowLongPtr(hwnd, GWLP_USERDATA, reinterpret_cast<LONG_PTR>(that));
return DefWindowProc(hwnd, WM_NCCREATE, wParam, lParam);
}
#pragma warning(suppress : 26490) // Win32 APIs can only store void*, have to use reinterpret_cast
auto terminal = reinterpret_cast<HwndTerminal*>(GetWindowLongPtr(hwnd, GWLP_USERDATA));
@@ -182,7 +189,7 @@ HwndTerminal::HwndTerminal(HWND parentHwnd) :
if (RegisterTermClass(hInstance))
{
_hwnd = wil::unique_hwnd(CreateWindowExW(
CreateWindowExW(
0,
term_window_class,
nullptr,
@@ -197,10 +204,7 @@ HwndTerminal::HwndTerminal(HWND parentHwnd) :
parentHwnd,
nullptr,
hInstance,
nullptr));
#pragma warning(suppress : 26490) // Win32 APIs can only store void*, have to use reinterpret_cast
SetWindowLongPtr(_hwnd.get(), GWLP_USERDATA, reinterpret_cast<LONG_PTR>(this));
this);
}
}
@@ -324,14 +328,15 @@ IRawElementProviderSimple* HwndTerminal::_GetUiaProvider() noexcept
{
// If TermControlUiaProvider throws during construction,
// we don't want to try constructing an instance again and again.
// _uiaProviderInitialized helps us prevent this.
if (!_uiaProviderInitialized)
if (!_uiaProvider)
{
try
{
auto lock = _terminal->LockForWriting();
LOG_IF_FAILED(::Microsoft::WRL::MakeAndInitialize<::Microsoft::Terminal::TermControlUiaProvider>(&_uiaProvider, this->GetUiaData(), this));
_uiaProviderInitialized = true;
LOG_IF_FAILED(::Microsoft::WRL::MakeAndInitialize<HwndTerminalAutomationPeer>(&_uiaProvider, this->GetUiaData(), this));
_uiaEngine = std::make_unique<::Microsoft::Console::Render::UiaEngine>(_uiaProvider.Get());
LOG_IF_FAILED(_uiaEngine->Enable());
_renderer->AddRenderEngine(_uiaEngine.get());
}
catch (...)
{
@@ -380,29 +385,10 @@ void HwndTerminal::SendOutput(std::wstring_view data)
HRESULT _stdcall CreateTerminal(HWND parentHwnd, _Out_ void** hwnd, _Out_ void** terminal)
{
// In order for UIA to hook up properly there needs to be a "static" window hosting the
// inner win32 control. If the static window is not present then WM_GETOBJECT messages
// will not reach the child control, and the uia element will not be present in the tree.
auto _hostWindow = CreateWindowEx(
0,
L"static",
nullptr,
WS_CHILD |
WS_CLIPCHILDREN |
WS_CLIPSIBLINGS |
WS_VISIBLE,
0,
0,
0,
0,
parentHwnd,
nullptr,
nullptr,
nullptr);
auto _terminal = std::make_unique<HwndTerminal>(_hostWindow);
auto _terminal = std::make_unique<HwndTerminal>(parentHwnd);
RETURN_IF_FAILED(_terminal->Initialize());
*hwnd = _hostWindow;
*hwnd = _terminal->GetHwnd();
*terminal = _terminal.release();
return S_OK;
@@ -725,6 +711,10 @@ try
{
modifiers |= ControlKeyStates::EnhancedKey;
}
if (vkey && keyDown && _uiaProvider)
{
_uiaProvider->RecordKeyEvent(vkey);
}
_terminal->SendKeyEvent(vkey, scanCode, modifiers, keyDown);
}
CATCH_LOG();
@@ -833,12 +823,20 @@ void __stdcall TerminalSetFocus(void* terminal)
{
auto publicTerminal = static_cast<HwndTerminal*>(terminal);
publicTerminal->_focused = true;
if (auto uiaEngine = publicTerminal->_uiaEngine.get())
{
LOG_IF_FAILED(uiaEngine->Enable());
}
}
void __stdcall TerminalKillFocus(void* terminal)
{
auto publicTerminal = static_cast<HwndTerminal*>(terminal);
publicTerminal->_focused = false;
if (auto uiaEngine = publicTerminal->_uiaEngine.get())
{
LOG_IF_FAILED(uiaEngine->Disable());
}
}
// Routine Description:

8
src/cascadia/PublicTerminalCore/HwndTerminal.hpp
View File

@@ -5,10 +5,10 @@
#include "../../renderer/base/Renderer.hpp"
#include "../../renderer/dx/DxRenderer.hpp"
#include "../../renderer/uia/UiaRenderer.hpp"
#include "../../cascadia/TerminalCore/Terminal.hpp"
#include <UIAutomationCore.h>
#include "../../types/IControlAccessibilityInfo.h"
#include "../../types/TermControlUiaProvider.hpp"
#include "HwndTerminalAutomationPeer.hpp"
using namespace Microsoft::Console::VirtualTerminal;
@@ -74,15 +74,15 @@ private:
FontInfo _actualFont;
int _currentDpi;
std::function<void(wchar_t*)> _pfnWriteCallback;
::Microsoft::WRL::ComPtr<::Microsoft::Terminal::TermControlUiaProvider> _uiaProvider;
::Microsoft::WRL::ComPtr<HwndTerminalAutomationPeer> _uiaProvider;
std::unique_ptr<::Microsoft::Terminal::Core::Terminal> _terminal;
std::unique_ptr<::Microsoft::Console::Render::Renderer> _renderer;
std::unique_ptr<::Microsoft::Console::Render::DxEngine> _renderEngine;
std::unique_ptr<::Microsoft::Console::Render::UiaEngine> _uiaEngine;
bool _focused{ false };
bool _uiaProviderInitialized{ false };
std::chrono::milliseconds _multiClickTime;
unsigned int _multiClickCounter{};

159
src/cascadia/PublicTerminalCore/HwndTerminalAutomationPeer.cpp Normal file
View File

@@ -0,0 +1,159 @@
// Copyright (c) Microsoft Corporation.
// Licensed under the MIT license.
#include "pch.h"
#include "HwndTerminalAutomationPeer.hpp"
#include "../../types/UiaTracing.h"
#include <UIAutomationCoreApi.h>
#pragma warning(suppress : 4471) // We don't control UIAutomationClient
#include <UIAutomationClient.h>
using namespace Microsoft::Console::Types;
static constexpr wchar_t UNICODE_NEWLINE{ L'\n' };
// Method Description:
// - creates a copy of the provided text with all of the control characters removed
// Arguments:
// - text: the string we're sanitizing
// Return Value:
// - a copy of "sanitized" with all of the control characters removed
static std::wstring Sanitize(std::wstring_view text)
{
std::wstring sanitized{ text };
sanitized.erase(std::remove_if(sanitized.begin(), sanitized.end(), [](wchar_t c) {
return (c < UNICODE_SPACE && c != UNICODE_NEWLINE) || c == 0x7F /*DEL*/;
}),
sanitized.end());
return sanitized;
}
// Method Description:
// - verifies if a given string has text that would be read by a screen reader.
// - a string of control characters, for example, would not be read.
// Arguments:
// - text: the string we're validating
// Return Value:
// - true, if the text is readable. false, otherwise.
static constexpr bool IsReadable(std::wstring_view text)
{
for (const auto c : text)
{
if (c > UNICODE_SPACE)
{
return true;
}
}
return false;
}
void HwndTerminalAutomationPeer::RecordKeyEvent(const WORD vkey)
{
if (const auto charCode{ MapVirtualKey(vkey, MAPVK_VK_TO_CHAR) })
{
if (const auto keyEventChar{ gsl::narrow_cast<wchar_t>(charCode) }; IsReadable({ &keyEventChar, 1 }))
{
_keyEvents.emplace_back(keyEventChar);
}
}
}
// Implementation of IRawElementProviderSimple::get_PropertyValue.
// Gets custom properties.
IFACEMETHODIMP HwndTerminalAutomationPeer::GetPropertyValue(_In_ PROPERTYID propertyId,
_Out_ VARIANT* pVariant) noexcept
{
pVariant->vt = VT_EMPTY;
// Returning the default will leave the property as the default
// so we only really need to touch it for the properties we want to implement
if (propertyId == UIA_ClassNamePropertyId)
{
// IMPORTANT: Do NOT change the name. Screen readers like may be dependent on this being "WpfTermControl".
pVariant->bstrVal = SysAllocString(L"WPFTermControl");
if (pVariant->bstrVal != nullptr)
{
pVariant->vt = VT_BSTR;
}
}
else
{
// fall back to shared implementation
return TermControlUiaProvider::GetPropertyValue(propertyId, pVariant);
}
return S_OK;
}
// Method Description:
// - Signals the ui automation client that the terminal's selection has changed and should be updated
// Arguments:
// - <none>
// Return Value:
// - <none>
void HwndTerminalAutomationPeer::SignalSelectionChanged()
{
UiaTracing::Signal::SelectionChanged();
LOG_IF_FAILED(UiaRaiseAutomationEvent(this, UIA_Text_TextSelectionChangedEventId));
}
// Method Description:
// - Signals the ui automation client that the terminal's output has changed and should be updated
// Arguments:
// - <none>
// Return Value:
// - <none>
void HwndTerminalAutomationPeer::SignalTextChanged()
{
UiaTracing::Signal::TextChanged();
LOG_IF_FAILED(UiaRaiseAutomationEvent(this, UIA_Text_TextChangedEventId));
}
// Method Description:
// - Signals the ui automation client that the cursor's state has changed and should be updated
// Arguments:
// - <none>
// Return Value:
// - <none>
void HwndTerminalAutomationPeer::SignalCursorChanged()
{
UiaTracing::Signal::CursorChanged();
LOG_IF_FAILED(UiaRaiseAutomationEvent(this, UIA_Text_TextSelectionChangedEventId));
}
void HwndTerminalAutomationPeer::NotifyNewOutput(std::wstring_view newOutput)
{
// Try to suppress any events (or event data)
// that is just the keypress the user made
auto sanitized{ Sanitize(newOutput) };
while (!_keyEvents.empty() && IsReadable(sanitized))
{
if (til::toupper_ascii(sanitized.front()) == _keyEvents.front())
{
// the key event's character (i.e. the "A" key) matches
// the output character (i.e. "a" or "A" text).
// We can assume that the output character resulted from
// the pressed key, so we can ignore it.
sanitized = sanitized.substr(1);
_keyEvents.pop_front();
}
else
{
// The output doesn't match,
// so clear the input stack and
// move on to fire the event.
_keyEvents.clear();
break;
}
}
// Suppress event if the remaining text is not readable
if (!IsReadable(sanitized))
{
return;
}
const auto sanitizedBstr = wil::make_bstr_nothrow(sanitized.c_str());
static auto activityId = wil::make_bstr_nothrow(L"TerminalTextOutput");
LOG_IF_FAILED(UiaRaiseNotificationEvent(this, NotificationKind_ActionCompleted, NotificationProcessing_All, sanitizedBstr.get(), activityId.get()));
}

42
src/cascadia/PublicTerminalCore/HwndTerminalAutomationPeer.hpp Normal file
View File

@@ -0,0 +1,42 @@
/*++
Copyright (c) Microsoft Corporation
Licensed under the MIT license.
Module Name:
- HwndTerminalAutomationPeer.hpp
Abstract:
- This module provides UI Automation access to the HwndTerminal
to support both automation tests and accessibility (screen
reading) applications. This mainly interacts with TermControlUiaProvider
to allow for shared code with Windows Terminal accessibility providers.
Author(s):
- Carlos Zamora (CaZamor) 2022
--*/
#pragma once
#include "../types/TermControlUiaProvider.hpp"
#include "../types/IUiaEventDispatcher.h"
#include "../types/IControlAccessibilityInfo.h"
class HwndTerminalAutomationPeer :
public ::Microsoft::Console::Types::IUiaEventDispatcher,
public ::Microsoft::Terminal::TermControlUiaProvider
{
public:
void RecordKeyEvent(const WORD vkey);
IFACEMETHODIMP GetPropertyValue(_In_ PROPERTYID idProp,
_Out_ VARIANT* pVariant) noexcept override;
#pragma region IUiaEventDispatcher
void SignalSelectionChanged() override;
void SignalTextChanged() override;
void SignalCursorChanged() override;
void NotifyNewOutput(std::wstring_view newOutput) override;
#pragma endregion
private:
std::deque<wchar_t> _keyEvents;
};

7
src/cascadia/PublicTerminalCore/PublicTerminalCore.vcxproj
View File

@@ -15,9 +15,11 @@
<PrecompiledHeader>Create</PrecompiledHeader>
</ClCompile>
<ClCompile Include="HwndTerminal.cpp" />
<ClCompile Include="HwndTerminalAutomationPeer.cpp" />
</ItemGroup>
<ItemGroup>
<ClInclude Include="HwndTerminal.hpp" />
<ClInclude Include="HwndTerminalAutomationPeer.hpp" />
<ClInclude Include="pch.h" />
</ItemGroup>
<ItemGroup>
@@ -39,6 +41,9 @@
<ProjectReference Include="$(SolutionDir)src\renderer\dx\lib\dx.vcxproj">
<Project>{48d21369-3d7b-4431-9967-24e81292cf62}</Project>
</ProjectReference>
<ProjectReference Include="..\..\renderer\uia\lib\uia.vcxproj">
<Project>{48d21369-3d7b-4431-9967-24e81292cf63}</Project>
</ProjectReference>
</ItemGroup>
<ItemDefinitionGroup>
<ClCompile>
@@ -55,4 +60,4 @@
<AdditionalDependencies>Uiautomationcore.lib;onecoreuap.lib;%(AdditionalDependencies)</AdditionalDependencies>
</Link>
</ItemDefinitionGroup>
</Project>
</Project>

6
src/cascadia/PublicTerminalCore/PublicTerminalCore.vcxproj.filters
View File

@@ -24,6 +24,9 @@
<ClCompile Include="HwndTerminal.cpp">
<Filter>Source Files</Filter>
</ClCompile>
<ClCompile Include="HwndTerminalAutomationPeer.cpp">
<Filter>Source Files</Filter>
</ClCompile>
</ItemGroup>
<ItemGroup>
<ClInclude Include="pch.h">
@@ -32,5 +35,8 @@
<ClInclude Include="HwndTerminal.hpp">
<Filter>Header Files</Filter>
</ClInclude>
<ClInclude Include="HwndTerminalAutomationPeer.hpp">
<Filter>Header Files</Filter>
</ClInclude>
</ItemGroup>
</Project>

1
src/cascadia/PublicTerminalCore/pch.h
View File

@@ -9,3 +9,4 @@
#endif
#include <LibraryIncludes.h>
#include <UIAutomationCore.h>

26
src/cascadia/WpfTerminalControl/TerminalContainer.cs
View File

@@ -8,6 +8,8 @@ namespace Microsoft.Terminal.Wpf
using System;
using System.Runtime.InteropServices;
using System.Windows;
using System.Windows.Automation;
using System.Windows.Automation.Peers;
using System.Windows.Interop;
using System.Windows.Media;
using System.Windows.Threading;
@@ -52,6 +54,30 @@ namespace Microsoft.Terminal.Wpf
}
}
/// <summary>
/// WPF's HwndHost likes to mark the WM_GETOBJECT message as handled to
/// force the usage of the WPF automation peer. We explicitly mark it as
/// not handled and don't return an automation peer in "OnCreateAutomationPeer" below.
/// This forces the message to go down to the HwndTerminal where we return terminal's UiaProvider.
/// </summary>
/// <inheritdoc/>
protected override IntPtr WndProc(IntPtr hwnd, int msg, IntPtr wParam, IntPtr lParam, ref bool handled)
{
if (msg == (int)NativeMethods.WindowMessage.WM_GETOBJECT)
{
handled = false;
return IntPtr.Zero;
}
return base.WndProc(hwnd, msg, wParam, lParam, ref handled);
}
/// <inheritdoc/>
protected override AutomationPeer OnCreateAutomationPeer()
{
return null;
}
/// <summary>
/// Event that is fired when the terminal buffer scrolls from text output.
/// </summary>

32
src/cascadia/WpfTerminalControl/TerminalControl.xaml.cs
View File

@@ -8,6 +8,7 @@ namespace Microsoft.Terminal.Wpf
using System;
using System.Threading;
using System.Windows;
using System.Windows.Automation.Peers;
using System.Windows.Controls;
using System.Windows.Input;
using System.Windows.Media;
@@ -42,6 +43,20 @@ namespace Microsoft.Terminal.Wpf
this.GotFocus += this.TerminalControl_GotFocus;
}
/// <inheritdoc/>
protected override AutomationPeer OnCreateAutomationPeer()
{
var peer = FrameworkElementAutomationPeer.FromElement(this);
if (peer == null)
{
// Provide our own automation peer here that just sets IsContentElement/IsControlElement to false
// (aka AccessibilityView = Raw). This makes it not pop up in the UIA tree.
peer = new TermControlAutomationPeer(this);
}
return peer;
}
/// <summary>
/// Gets the current character rows available to the terminal.
/// </summary>
@@ -266,5 +281,22 @@ namespace Microsoft.Terminal.Wpf
var viewTop = (int)e.NewValue;
this.termContainer.UserScroll(viewTop);
}
private class TermControlAutomationPeer : UserControlAutomationPeer
{
public TermControlAutomationPeer(UserControl owner) : base(owner)
{
}
protected override bool IsContentElementCore()
{
return false;
}
protected override bool IsControlElementCore()
{
return false;
}
}
}
}

46
src/types/TermControlUiaProvider.cpp
View File

@@ -19,6 +19,52 @@ HRESULT TermControlUiaProvider::RuntimeClassInitialize(_In_ ::Microsoft::Console
return S_OK;
}
// Implementation of IRawElementProviderSimple::get_PropertyValue.
// Gets custom properties.
IFACEMETHODIMP TermControlUiaProvider::GetPropertyValue(_In_ PROPERTYID propertyId,
_Out_ VARIANT* pVariant) noexcept
{
pVariant->vt = VT_EMPTY;
// Returning the default will leave the property as the default
// so we only really need to touch it for the properties we want to implement
switch (propertyId)
{
case UIA_ClassNamePropertyId:
pVariant->bstrVal = SysAllocString(L"TermControl");
if (pVariant->bstrVal != nullptr)
{
pVariant->vt = VT_BSTR;
}
break;
case UIA_ControlTypePropertyId:
pVariant->vt = VT_I4;
pVariant->lVal = UIA_TextControlTypeId;
break;
case UIA_LocalizedControlTypePropertyId:
// TODO: we should use RS_(L"TerminalControl_ControlType"),
// but that's exposed/defined in the TermControl project
pVariant->bstrVal = SysAllocString(L"terminal");
if (pVariant->bstrVal != nullptr)
{
pVariant->vt = VT_BSTR;
}
break;
case UIA_OrientationPropertyId:
pVariant->vt = VT_I4;
pVariant->lVal = OrientationType_Vertical;
break;
case UIA_LiveSettingPropertyId:
pVariant->vt = VT_I4;
pVariant->lVal = LiveSetting::Polite;
break;
default:
// fall back to the shared implementation
return ScreenInfoUiaProviderBase::GetPropertyValue(propertyId, pVariant);
}
return S_OK;
}
IFACEMETHODIMP TermControlUiaProvider::Navigate(_In_ NavigateDirection direction,
_COM_Outptr_result_maybenull_ IRawElementProviderFragment** ppProvider) noexcept
{

4
src/types/TermControlUiaProvider.hpp
View File

@@ -32,6 +32,10 @@ namespace Microsoft::Terminal
HRESULT RuntimeClassInitialize(_In_ ::Microsoft::Console::Types::IUiaData* const uiaData,
_In_ ::Microsoft::Console::Types::IControlAccessibilityInfo* controlInfo) noexcept;
// IRawElementProviderSimple methods
IFACEMETHODIMP GetPropertyValue(_In_ PROPERTYID idProp,
_Out_ VARIANT* pVariant) noexcept override;
// IRawElementProviderFragment methods
IFACEMETHODIMP Navigate(_In_ NavigateDirection direction,
_COM_Outptr_result_maybenull_ IRawElementProviderFragment** ppProvider) noexcept override;