James Holderness dccc1f4240 Refactor VT terminal input (#16511)
The primary reason for this refactoring was to simplify the management
of VT input sequences that vary depending on modes, adding support for
the missing application keypad sequences, and preparing the way for
future extensions like `S8C1T`.

However, it also includes fixes for a number of keyboard related bugs,
including a variety of missing or incorrect mappings for the `Ctrl` and
`Ctrl`+`Alt` key combinations, 

## References and Relevant Issues

This PR also includes a fix for #10308, which was previously closed as a
duplicate of #10551. I don't think those bugs were related, though, and
although they're both supposed to be fixed in Windows 11, this PR fixes
the issue in Windows 10.

## Detailed Description of the Pull Request / Additional comments

The way the input now works, there's a single keyboard map that takes a
virtual key code combined with `Ctrl`, `Alt`, and `Shift` modifier bits
as the lookup key, and the expected VT input sequence as the value. This
map is initially constructed at startup, and then regenerated whenever a
keyboard mode is changed.

This map takes care of the cursor keys, editing keys, function keys, and
keys like `BkSp` and `Return` which can be affected by mode changes. The
remaining "graphic" key combinations are determined manually at the time
of input.

The order of precedence looks like this:

1. If the virtual key is `0` or `VK_PACKET`, it's considered to be a
   synthesized keyboard event, and the `UnicodeChar` value is used
   exactly as given.

2. If it's a numeric keypad key, and `Alt` is pressed (but not `Ctrl`),
   then it's assumedly part of an Alt-Numpad composition, so the key
   press is ignored (the generated character will be transmitted when
   the `Alt` is released).

3. If the virtual key combined with modifier bits is found in the key
   map described above, then the matched escape sequence will be used
   used as the output.

4. If a `UnicodeChar` value has been provided, that will be used as the
   output, but possibly with additional Ctrl and Alt modifiers applied:

   a. If it's an `AltGr` key, and we've got either two `Ctrl` keys
      pressed or a left `Ctrl` key that is distinctly separate from a
      right `Alt` key, then we will try and convert the character into
      a C0 control code.

   b. If an `Alt` key is pressed (or in the case of an `AltGr` value,
      both `Alt` keys are pressed), then we will convert it into an
      Alt-key sequence by prefixing the character with an `ESC`.

5. If we don't have a `UnicodeChar`, we'll use the `ToUnicodeEx` API to
   check whether the current keyboard state reflects a dead key, and if
   so, return nothing.

6. Otherwise we'll make another `ToUnicodeEx` call but with any `Ctrl`
   and `Alt` modifiers removed from the state to determine the base key
   value. Once we have that, we can apply the modifiers ourself.

   a. If the `Ctrl` key is pressed, we'll try and convert the base value
      into a C0 control code. But if we can't do that, we'll try again
      with the virtual key code (if it's alphanumeric) as a fallback.

   b. If the `Alt` key is pressed, we'll convert the base value (or
      control code value) into an Alt-key sequence by prefixing it with
      an `ESC`.

For step 4-a, we determine whether the left `Ctrl` key is distinctly
separate from the right `Alt` key by recording the time that those keys
are pressed, and checking for a time gap greater than 50ms. This is
necessary to distinguish between the user pressing `Ctrl`+`AltGr`, or
just pressing `AltGr` alone, which triggers a fake `Ctrl` key press at
the same time.

## Validation Steps Performed

I created a test script to automate key presses in the terminal window
for every relevant key, along with every Ctrl/Alt/Shift modifier, and
every relevant mode combination. I then compared the generated input
sequences with XTerm and a DEC VT240 terminal. The idea wasn't to match
either of them exactly, but to make sure the places where we differed
were intentional and reasonable.

This mostly dealt with the US keyboard layout. Comparing international
layouts wasn't really feasible because DEC, Linux, and Windows keyboard
assignments tend to be quite different. However, I've manually tested a
number of different layouts, and tried to make sure that they were all
working in a reasonable manner.

In terms of unit testing, I haven't done much more than patching the
ones that already existed to get them to pass. They're honestly not
great tests, because they aren't generating events in the form that
you'd expect for a genuine key press, and that can significantly affect
the results, but I can't think of an easy way to improve them.

## PR Checklist
- [x] Closes #16506
- [x] Closes #16508
- [x] Closes #16509
- [x] Closes #16510
- [x] Closes #3483
- [x] Closes #11194
- [x] Closes #11700
- [x] Closes #12555
- [x] Closes #13319
- [x] Closes #15367
- [x] Closes #16173
- [x] Tests added/passed
2024-01-29 16:58:39 -08:00
2023-12-05 15:40:23 -08:00
2023-12-05 15:40:23 -08:00
2023-08-23 11:17:11 -05:00
2024-01-29 16:58:39 -08:00
2023-09-25 13:40:13 -05:00
2021-12-15 14:56:36 +00:00

terminal-logos

Welcome to the Windows Terminal, Console and Command-Line repo

This repository contains the source code for:

Related repositories include:

Installing and running Windows Terminal

Note

Windows Terminal requires Windows 10 2004 (build 19041) or later

Install the Windows Terminal from the Microsoft Store. This allows you to always be on the latest version when we release new builds with automatic upgrades.

This is our preferred method.

Other install methods

Via GitHub

For users who are unable to install Windows Terminal from the Microsoft Store, released builds can be manually downloaded from this repository's Releases page.

Download the Microsoft.WindowsTerminal_<versionNumber>.msixbundle file from the Assets section. To install the app, you can simply double-click on the .msixbundle file, and the app installer should automatically run. If that fails for any reason, you can try the following command at a PowerShell prompt:

# NOTE: If you are using PowerShell 7+, please run
# Import-Module Appx -UseWindowsPowerShell
# before using Add-AppxPackage.

Add-AppxPackage Microsoft.WindowsTerminal_<versionNumber>.msixbundle

Note

If you install Terminal manually:

  • You may need to install the VC++ v14 Desktop Framework Package. This should only be necessary on older builds of Windows 10 and only if you get an error about missing framework packages.
  • Terminal will not auto-update when new builds are released so you will need to regularly install the latest Terminal release to receive all the latest fixes and improvements!

Via Windows Package Manager CLI (aka winget)

winget users can download and install the latest Terminal release by installing the Microsoft.WindowsTerminal package:

winget install --id Microsoft.WindowsTerminal -e

Note

Dependency support is available in WinGet version 1.6.2631 or later. To install the Terminal stable release 1.18 or later, please make sure you have the updated version of the WinGet client.

Via Chocolatey (unofficial)

Chocolatey users can download and install the latest Terminal release by installing the microsoft-windows-terminal package:

choco install microsoft-windows-terminal

To upgrade Windows Terminal using Chocolatey, run the following:

choco upgrade microsoft-windows-terminal

If you have any issues when installing/upgrading the package please go to the Windows Terminal package page and follow the Chocolatey triage process

Via Scoop (unofficial)

Scoop users can download and install the latest Terminal release by installing the windows-terminal package:

scoop bucket add extras
scoop install windows-terminal

To update Windows Terminal using Scoop, run the following:

scoop update windows-terminal

If you have any issues when installing/updating the package, please search for or report the same on the issues page of Scoop Extras bucket repository.


Installing Windows Terminal Canary

Windows Terminal Canary is a nightly build of Windows Terminal. This build has the latest code from our main branch, giving you an opportunity to try features before they make it to Windows Terminal Preview.

Windows Terminal Canary is our least stable offering, so you may discover bugs before we have had a chance to find them.

Windows Terminal Canary is available as an App Installer distribution and a Portable ZIP distribution.

The App Installer distribution supports automatic updates. Due to platform limitations, this installer only works on Windows 11.

The Portable ZIP distribution is a portable application. It will not automatically update and will not automatically check for updates. This portable ZIP distribution works on Windows 10 (19041+) and Windows 11.

Distribution Architecture Link
App Installer x64, arm64, x86 download
Portable ZIP x64 download
Portable ZIP ARM64 download
Portable ZIP x86 download

Learn more about the types of Windows Terminal distributions.


Windows Terminal Roadmap

The plan for the Windows Terminal is described here and will be updated as the project proceeds.

Project Build Status

Project Build Status
Terminal Terminal Build Status
ColorTool Colortool Build Status

Terminal & Console Overview

Please take a few minutes to review the overview below before diving into the code:

Windows Terminal

Windows Terminal is a new, modern, feature-rich, productive terminal application for command-line users. It includes many of the features most frequently requested by the Windows command-line community including support for tabs, rich text, globalization, configurability, theming & styling, and more.

The Terminal will also need to meet our goals and measures to ensure it remains fast and efficient, and doesn't consume vast amounts of memory or power.

The Windows Console Host

The Windows Console host, conhost.exe, is Windows' original command-line user experience. It also hosts Windows' command-line infrastructure and the Windows Console API server, input engine, rendering engine, user preferences, etc. The console host code in this repository is the actual source from which the conhost.exe in Windows itself is built.

Since taking ownership of the Windows command-line in 2014, the team added several new features to the Console, including background transparency, line-based selection, support for ANSI / Virtual Terminal sequences, 24-bit color, a Pseudoconsole ("ConPTY"), and more.

However, because Windows Console's primary goal is to maintain backward compatibility, we have been unable to add many of the features the community (and the team) have been wanting for the last several years including tabs, unicode text, and emoji.

These limitations led us to create the new Windows Terminal.

You can read more about the evolution of the command-line in general, and the Windows command-line specifically in this accompanying series of blog posts on the Command-Line team's blog.

Shared Components

While overhauling Windows Console, we modernized its codebase considerably, cleanly separating logical entities into modules and classes, introduced some key extensibility points, replaced several old, home-grown collections and containers with safer, more efficient STL containers, and made the code simpler and safer by using Microsoft's Windows Implementation Libraries - WIL.

This overhaul resulted in several of Console's key components being available for re-use in any terminal implementation on Windows. These components include a new DirectWrite-based text layout and rendering engine, a text buffer capable of storing both UTF-16 and UTF-8, a VT parser/emitter, and more.

Creating the new Windows Terminal

When we started planning the new Windows Terminal application, we explored and evaluated several approaches and technology stacks. We ultimately decided that our goals would be best met by continuing our investment in our C++ codebase, which would allow us to reuse several of the aforementioned modernized components in both the existing Console and the new Terminal. Further, we realized that this would allow us to build much of the Terminal's core itself as a reusable UI control that others can incorporate into their own applications.

The result of this work is contained within this repo and delivered as the Windows Terminal application you can download from the Microsoft Store, or directly from this repo's releases.


Resources

For more information about Windows Terminal, you may find some of these resources useful and interesting:


FAQ

I built and ran the new Terminal, but it looks just like the old console

Cause: You're launching the incorrect solution in Visual Studio.

Solution: Make sure you're building & deploying the CascadiaPackage project in Visual Studio.

Note

OpenConsole.exe is just a locally-built conhost.exe, the classic Windows Console that hosts Windows' command-line infrastructure. OpenConsole is used by Windows Terminal to connect to and communicate with command-line applications (via ConPty).


Documentation

All project documentation is located at aka.ms/terminal-docs. If you would like to contribute to the documentation, please submit a pull request on the Windows Terminal Documentation repo.


Contributing

We are excited to work alongside you, our amazing community, to build and enhance Windows Terminal!

BEFORE you start work on a feature/fix, please read & follow our Contributor's Guide to help avoid any wasted or duplicate effort.

Communicating with the Team

The easiest way to communicate with the team is via GitHub issues.

Please file new issues, feature requests and suggestions, but DO search for similar open/closed preexisting issues before creating a new issue.

If you would like to ask a question that you feel doesn't warrant an issue (yet), please reach out to us via Twitter:

Developer Guidance

Prerequisites

Building the Code

This repository uses git submodules for some of its dependencies. To make sure submodules are restored or updated, be sure to run the following prior to building:

git submodule update --init --recursive

OpenConsole.sln may be built from within Visual Studio or from the command-line using a set of convenience scripts & tools in the /tools directory:

Building in PowerShell

Import-Module .\tools\OpenConsole.psm1
Set-MsBuildDevEnvironment
Invoke-OpenConsoleBuild

Building in Cmd

.\tools\razzle.cmd
bcz

Running & Debugging

To debug the Windows Terminal in VS, right click on CascadiaPackage (in the Solution Explorer) and go to properties. In the Debug menu, change "Application process" and "Background task process" to "Native Only".

You should then be able to build & debug the Terminal project by hitting F5. Make sure to select either the "x64" or the "x86" platform - the Terminal doesn't build for "Any Cpu" (because the Terminal is a C++ application, not a C# one).

👉 You will not be able to launch the Terminal directly by running the WindowsTerminal.exe. For more details on why, see #926, #4043

Coding Guidance

Please review these brief docs below about our coding practices.

👉 If you find something missing from these docs, feel free to contribute to any of our documentation files anywhere in the repository (or write some new ones!)

This is a work in progress as we learn what we'll need to provide people in order to be effective contributors to our project.


Code of Conduct

This project has adopted the Microsoft Open Source Code of Conduct. For more information see the Code of Conduct FAQ or contact opencode@microsoft.com with any additional questions or comments.

Languages
C++ 92.8%
C# 4%
C 1.5%
PowerShell 0.8%
PHP 0.3%
Other 0.4%