## Summary of the Pull Request
In pursuit of reflowing the terminal buffer on resize, move the reflow algorithm to the TextBuffer. This does _not_ yet add support for reflowing in the Windows Terminal.
## References
## PR Checklist
* [ ] There's not really an issue for this yet, I'm just breaking this work up into as many PRs as possible to help the inevitable bisect.
* [x] I work here
* [x] Ideally, all the existing tests will pass
* [n/a] Requires documentation to be updated
## Detailed Description of the Pull Request / Additional comments
In `SCREEN_INFORMATION::ResizeScreenBuffer`, the screenbuffer needs to create a new buffer, and copy the contents of the old buffer into the new one. I'm moving that "copy contents from the old buffer to the new one" step to it's own helper, as a static function on `TextBuffer`. That way, when the time comes to implement this for the Terminal, the hard part of the code will already be there.
## Validation Steps Performed
Ideally, all the tests will still pass.
## Summary of the Pull Request
When `GenHTML` or `GenRTF` encountered an empty line, they assumed that `CR` is the last character of the row and wrote it, even though in general `CR` and `LF` just break the line and instead of them either `<BR>` in HTML or `\line` in RTF is written. Don't know how I missed that in #2038.
Another question is whether the `TextAndColor` structure which these methods receive and which is generated by `TextBuffer::GetTextForClipboard` should really contain `\r\n` at the end of each row. I think it'd be cleaner if it didn't esp. that afaik these last 2 characters don't have associated valid color information.
## References
<!-- Please review the items on the PR checklist before submitting-->
## PR Checklist
* [X] Closes#4187
* [X] CLA signed. If not, go over [here](https://cla.opensource.microsoft.com/microsoft/Terminal) and sign the CLA
* [ ] Tests added/passed - there aren't any related tests, right?
* [ ] Requires documentation to be updated
* [ ] I've discussed this with core contributors already. If not checked, I'm ready to accept this work might be rejected in favor of a different grand plan. Issue number where discussion took place: #4147
<!-- Describe how you validated the behavior. Add automated tests wherever possible, but list manual validation steps taken as well -->
## Validation Steps Performed
Copied various terminal states and verified the generated HTML.
## Summary of the Pull Request
- Enables auditing of some Terminal libraries (Connection, Core, Settings)
- Also audit WinConPTY.LIB since Connection depends on it
## PR Checklist
* [x] Rolls audit out to more things
* [x] I work here
* [x] Tests should still pass
* [x] Am core contributor
## Detailed Description of the Pull Request / Additional comments
This is turning on the auditing of these projects (as enabled by the heavier lifting in the other refactor) and then cleaning up the remaining warnings.
## Validation Steps Performed
- [x] Built it
- [x] Ran the tests
<!-- Enter a brief description/summary of your PR here. What does it fix/what does it change/how was it tested (even manually, if necessary)? -->
This is the PR for feature Search: #605
This PR includes the newly introduced SearchBoxControl in TermControl dir, which is the search bar for the search experience. And the codes that enable Search in Windows Terminal.
<!-- Other than the issue solved, is this relevant to any other issues/existing PRs? -->
The PR that migrates the Conhost search module: https://github.com/microsoft/terminal/pull/3279
Spec (still actively updating): https://github.com/microsoft/terminal/pull/3299
<!-- Please review the items on the PR checklist before submitting-->
## PR Checklist
* [x] Closes#605
* [ ] CLA signed. If not, go over [here](https://cla.opensource.microsoft.com/microsoft/Terminal) and sign the CLA
* [ ] Tests added/passed
* [ ] Requires documentation to be updated
* [ ] I've discussed this with core contributors already. If not checked, I'm ready to accept this work might be rejected in favor of a different grand plan. Issue number where discussion took place: #xxx
<!-- Provide a more detailed description of the PR, other things fixed or any additional comments/features here -->
These functionalities are included in the search experience.
1. Search in Terminal text buffer.
2. Automatic wrap-around.
3. Search up or down switch by clicking different buttons.
4. Search case sensitively/insensitively by clicking a button. S. Move the search box to the top/bottom by clicking a button.
6. Close by clicking 'X'.
7. Open search by ctrl + F.
When the searchbox is open, the user could still interact with the terminal by clicking the terminal input area.
While I already have the search functionalities, currently there are still some known to-do works and I will keep updating my PR:
1. Optimize the search box UI, this includes:
1) Theme adaptation. The search box background and font color
should change according to the theme,
2) Add background. Currently the elements in search box are all
transparent. However, we need a background.
3) Move button should be highlighted once clicked.
2. Accessibility: search process should be able to performed without mouse. Once the search box is focused, the user should be able to navigate between all interactive elements on the searchbox using keyboard.
<!-- Describe how you validated the behavior. Add automated tests wherever possible, but list manual validation steps taken as well -->
To test:
1. checkout this branch.
2. Build the project.
3. Start Windows Terminal and press Ctrl+F
4. The search box should appear on the top right corner.
Enables support for word navigation when using an automation client (i.e.: Narrator, etc...). Specifically, adds this functionality to the UiaTextRange class. The only delimiter used is whitespace because that's how words are separated in English.
# General "Word Movement" Expectations
The resulting text range should include any word break characters that are present at the end of the word, but before the start of the next word. (Source)
If you already are on a word, getting the "next word" means you skip the word you are on, and highlight the upcoming word appropriately. (similar idea when moving backwards)
# Word Expansion
Since word selection is supposed to detect word delimiters already, I figured I'd reuse that code. I moved it from TerminalCore to the TextBuffer.
Then I built on top of it by adding an optional additional parameter that decides if you want to include...
- the delimiter run when moving forward
- the character run when moving backwards
It defaults to false so that we don't have to care when using it in selection. But we change it to true when using it in our UiaTextRange
# UiaTextRange
The code is based on character movement. This allows us to actually work with boundary conditions.
The main thing to remember here is that each text range is recorded as a MoveState. The text range is most easily defined when you think about the start Endpoint and the end Endpoint. An Endpoint is just a linear 1-dimensional indexing of the text buffer. Examples:
- Endpoint 0 --> (0,0)
- Endpoint 79 --> (79,0) (when the buffer width is 80)
- Endpoint 80 -->(0,1) (when the buffer width is 80)
- When moving forward, the strategy is to focus on moving the end Endpoint. That way, we properly get the indexing for the "next" word (this also fixes a wrapping issue). Then, we update the start Endpoint. (This is reversed for moving backwards).
- When moving a specific Endpoint, we just have a few extra if statements to properly adjust for moving start vs end.
# Hooking it up
All we really had to do is add an enum. This part was super easy :)
I originally wanted the delimiters to be able to be defined. I'm not so sure about that anymore. Either way, I hardcoded our delimiter into a variable so if we ever want to expand on it or make that customizable, we just modify that variable.
# Defining your own word delimiters
- Import a word delimiter into the constructor of the ScreenInfoUiaProvider (SIUP)
- This defines a word delimiter for all the UiaTextRanges (UTR) created by in this context
- import a word delimiter into the UTR directly
- this provides more control over what a "word" is
- this can be useful if you have an idea of what text a particular UTR will encounter and you want to customize the word navigation for it (i.e consider adding / or \\ for file paths)
The default param of " " is scattered throughout because this is the word delimiter used in the English language.
## Summary of the Pull Request
Operations that erase areas of the screen are typically meant to do so using the current color attributes, but with the rendition attributes reset (what we refer to as meta attributes). This also includes scroll operations that have to clear the area of the screen that has scrolled into view. The only exception is the _Erase Scrollback_ operation, which needs to reset the buffer with the default attributes. This PR updates all of these cases to apply the correct attributes when scrolling and erasing.
## PR Checklist
* [x] Closes#2553
* [x] CLA signed. If not, go over [here](https://cla.opensource.microsoft.com/microsoft/Terminal) and sign the CLA
* [x] Tests added/passed
* [ ] Requires documentation to be updated
* [ ] I've not really discussed this with core contributors. I'm ready to accept this work might be rejected in favor of a different grand plan.
## Detailed Description of the Pull Request / Additional comments
My initial plan was to use a special case legacy attribute value to indicate the "standard erase attribute" which could safely be passed through the legacy APIs. But this wouldn't cover the cases that required default attributes to be used. And then with the changes in PR #2668 and #2987, it became clear that our requirements could be better achieved with a couple of new private APIs that wouldn't have to depend on legacy attribute hacks at all.
To that end, I've added the `PrivateFillRegion` and `PrivateScrollRegion` APIs to the `ConGetSet` interface. These are just thin wrappers around the existing `SCREEN_INFORMATION::Write` method and the `ScrollRegion` function respectively, but with a simple boolean parameter to choose between filling with default attributes or the standard erase attributes (i.e the current colors but with meta attributes reset).
With those new APIs in place, I could then update most scroll operations to use `PrivateScrollRegion`, and most erase operations to use `PrivateFillRegion`.
The functions affected by scrolling included:
* `DoSrvPrivateReverseLineFeed` (the RI command)
* `DoSrvPrivateModifyLinesImpl` (the IL and DL commands)
* `AdaptDispatch::_InsertDeleteHelper` (the ICH and DCH commands)
* `AdaptDispatch::_ScrollMovement` (the SU and SD commands)
The functions affected by erasing included:
* `AdaptDispatch::_EraseSingleLineHelper` (the EL command, and most ED variants)
* `AdaptDispatch::EraseCharacters` (the ECH command)
While updating these erase methods, I noticed that both of them also required boundary fixes similar to those in PR #2505 (i.e. the horizontal extent of the erase operation should apply to the full width of the buffer, and not just the current viewport width), so I've addressed that at the same time.
In addition to the changes above, there were also a few special cases, the first being the line feed handling, which required updating in a number of places to use the correct erase attributes:
* `SCREEN_INFORMATION::InitializeCursorRowAttributes` - this is used to initialise the rows that pan into view when the viewport is moved down the buffer.
* `TextBuffer::IncrementCircularBuffer` - this occurs when we scroll passed the very end of the buffer, and a recycled row now needs to be reinitialised.
* `AdjustCursorPosition` - when within margin boundaries, this relies on a couple of direct calls to `ScrollRegion` which needed to be passed the correct fill attributes.
The second special case was the full screen erase sequence (`ESC 2 J`), which is handled separately from the other ED sequences. This required updating the `SCREEN_INFORMATION::VtEraseAll` method to use the standard erase attributes, and also required changes to the horizontal extent of the filled area, since it should have been clearing the full buffer width (the same issue as the other erase operations mentioned above).
Finally, there was the `AdaptDispatch::_EraseScrollback` method, which uses both scroll and fill operations, which could now be handled by the new `PrivateScrollRegion` and `PrivateFillRegion` APIs. But in this case we needed to fill with the default attributes rather than the standard erase attributes. And again this implementation needed some changes to make sure the full width of the active area was retained after the erase, similar to the horizontal boundary issues with the other erase operations.
Once all these changes were made, there were a few areas of the code that could then be simplified quite a bit. The `FillConsoleOutputCharacterW`, `FillConsoleOutputAttribute`, and `ScrollConsoleScreenBufferW` were no longer needed in the `ConGetSet` interface, so all of that code could now be removed. The `_EraseSingleLineDistanceHelper` and `_EraseAreaHelper` methods in the `AdaptDispatch` class were also no longer required and could be removed.
Then there were the hacks to handle legacy default colors in the `FillConsoleOutputAttributeImpl` and `ScrollConsoleScreenBufferWImpl` implementations. Since those hacks were only needed for VT operations, and the VT code no longer calls those methods, there was no longer a need to retain that behaviour (in fact there are probably some edge cases where that behaviour might have been considered a bug when reached via the public console APIs).
## Validation Steps Performed
For most of the scrolling operations there were already existing tests in place, and those could easily be extended to check that the meta attributes were correctly reset when filling the revealed lines of the scrolling region.
In the screen buffer tests, I made updates of that sort to the `ScrollOperations` method (handling SU, SD, IL, DL, and RI), the `InsertChars` and `DeleteChars` methods (ICH and DCH), and the `VtNewlinePastViewport` method (LF). I also added a new `VtNewlinePastEndOfBuffer` test to check the case where the line feed causes the viewport to pan past the end of the buffer.
The erase operations, however, were being covered by adapter tests, and those aren't really suited for this kind of functionality (the same sort of issue came up in PR #2505). As a result I've had to reimplement those tests as screen buffer tests.
Most of the erase operations are covered by the `EraseTests` method, except the for the scrollback erase which has a dedicated `EraseScrollbackTests` method. I've also had to replace the `HardReset` adapter test, but that was already mostly covered by the `HardResetBuffer` screen buffer test, which I've now extended slightly (it could do with some more checks, but I think that can wait for a future PR when we're fixing other RIS issues).
* Make search a shared component for conhost and terminal
* Remove inclusion of deprecated interface file
* Code review changes, remove text buffer modification in Terminal
* remove unreferenced objects to fix build errors
* Fix test failure, guarantee uiaData object is correctly initialized in Search
* minor comment typo fix and format fix
* minor PR comments change
* ColorSeclection directly throw and return
* remove coordAnchor initialization
* minor method signature change
## Summary of the Pull Request
RTF data is now copied to the clipboard. Tested by copy pasting text from terminal to WordPad.
<!-- Please review the items on the PR checklist before submitting-->
## PR Checklist
* [x] Closes#2487
* [x] CLA signed. If not, go over [here](https://cla.opensource.microsoft.com/microsoft/Terminal) and sign the CLA
* [ ] Tests added/passed
* [ ] Requires documentation to be updated
* [x] I've discussed this with core contributors already. If not checked, I'm ready to accept this work might be rejected in favor of a different grand plan. Issue number where discussion took place: #2487
<!-- Provide a more detailed description of the PR, other things fixed or any additional comments/features here -->
## Detailed Description of the Pull Request / Additional comments
Mostly similar to PR #1224. Added a new static method `GenRTF` in `TextBuffer` that is responsible
for generating the RTF representation of a given text. The generated RTF is added to the `DataPackage` that is ultimately passed to the clipboard.
<!-- Describe how you validated the behavior. Add automated tests wherever possible, but list manual validation steps taken as well -->
## Validation Steps Performed
Validated by copy pasting text from the terminal to WordPad. Validated with different colors to make sure that is working. (MS Word seems to prefer HTML data from the clipboard instead of RTF.)
<hr>
* Copy RTF data to the clipboard
* Added comment explaining various parts of the header
* Fixed static code analysis issues and added noexcept to GenRTF()
* Removed noexcept
This commit cleans up and deduplicates all of the common build
preamble/postamble across exe, dll, lib and c++/winrt projects.
The following specific changes have been made:
* All projects now define their ConfigurationType
* All projects now set all their properties *before* including a common
build file (or any other build files)
* cppwinrt.pre and cppwinrt.post now delegate most of their
configuration to common.pre and common.post
* (becuase of the above,) all build options are conserved between
console and c++/winrt components, including specific warnings and
preprocessor definitions.
* More properties that are configurable per-project are now
conditioned so the common props don't override them.
* The exe, dll, exe.or.dll, and lib postincludes have been merged into
pre or post and switched based on condition as required
* Shared items (-shared, -common) are now explicitly vcxitems instead of
vcxproj files.
* The link line is now manipulated after Microsoft.Cpp sets it, so the
libraries we specify "win". All console things link first against
onecore_apiset.lib.
* Fix all compilation errors caused by build unification
* Move CascadiaPackage's resources into a separate item file
Fixes#922.
We now truncate the font name as it goes out to GDI APIs, in console API
servicing, and in the propsheet.
I attempted to defer truncating the font to as far up the stack as
possible, so as to make FontInfo usable for the broadest set of cases.
There were a couple questions that came up: I know that `Settings` gets
memset (memsat?) by the registry deserializer, and perhaps that's
another place for us to tackle. Right now, this pull request enables
fonts whose names are >= 32 characters _in Windows Terminal only_, but
the underpinnings are there for conhost as well. We'd need to explicitly
break at the API, or perhaps return a failure or log something to
telemetry.
* Should we log truncation at the API boundary to telemetry?
-> Later; followup filed (#3123)
* Should we fix Settings here, or later?
-> Later; followup filed (#3123)
* `TrueTypeFontList` is built out of things in winconp, the private
console header. Concern about interop structures.
-> Not used for interop, followup filed to clean it up (#3123)
* Is `unsigned int` right for codepage? For width?
-> Yes: codepage became UINT (from WORD) when we moved from Win16 to
Win32
This commit also includes a workaround for #3170. Growing
CONSOLE_INFORMATION made us lose the struct layout lottery during
release builds, and this was an expedient fix.
Closes#602.
Related to #3123.
This adds the WPF control to our project, courtesy of the Visual Studio team.
It re-hosts the Terminal Control components inside a reusable WPF adapter so it can be composed onto C# type surfaces like Visual Studio requires.
## Summary of the Pull Request
Adds support for Italics, Blinking, Invisible, CrossedOut text, THROUGH CONPTY. This does **NOT** add support for those styles to conhost or the terminal.
We will store these "Extended Text Attributes" in a `TextAttribute`. When we go to render a line, we'll see if the state has changed from our previous state, and if so, we'll appropriately toggle that state with VT. Boldness has been moved from a `bool` to a single bit in these flags.
Technically, now that these are stored in the buffer, we only need to make changes to the renderers to be able to support them. That's not being done as a part of this PR however.
## References
See also #2915 and #2916, which are some follow-up tasks from this fix. I thought them too risky for 20H1.
## PR Checklist
* [x] Closes#2554
* [x] I work here
* [x] Tests added/passed
* [n/a] Requires documentation to be updated
<hr>
* store text with extended attributes too
* Plumb attributes through all the renderers
* parse extended attrs, though we're not renderering them right
* Render these states correctly
* Add a very extensive test
* Cleanup for PR
* a block of PR feedback
* add 512 test cases
* Fix the build
* Fix @carlos-zamora's suggestions
* @miniksa's PR feedback
EraseInLine calls `FillConsoleOutputCharacterW()`. In filling the row with
chars, we were setting the wrap flag. We need to specifically not do this on
ANY _FILL_ operation. Now a fill operation UNSETS the wrap flag if we fill to
the end of the line.
Originally, we had a boolean `setWrap` that would mean...
- **true**: if writing to the end of the row, SET the wrap value to true
- **false**: if writing to the end of the row, DON'T CHANGE the wrap value
Now we're making this bool a std::optional to allow for a ternary state. This
allows for us to handle the following cases completely. Refer to the table
below:
,- current wrap value
| ,- are we filling the last cell in the row?
| | ,- new wrap value
| | | ,- comments
|-- |-- |-- |
| 0 | 0 | 0 |
| 0 | 1 | 0 |
| 0 | 1 | 1 | THIS CASE WAS HANDLED CORRECTLY
| 1 | 0 | 0 | THIS CASE WAS UNHANDLED
| 1 | 0 | 1 |
| 1 | 1 | 1 |
To handle that special case (1-0-0), we need to UNSET the wrap. So now, we have
~setWrap~ `wrap` mean the following:
- **true**: if writing to the end of the row, SET the wrap value to TRUE
- **false**: if writing to the end of the row, SET the wrap value to FALSE
- **nullopt**: leave the wrap value as it is
Closes#1126
* Move Clipboard::GenHTML to TextBuffer (add params)
Refactor RetrieveSelectedTextFromBuffer
Modify CopyToClipboardEventArgs to include HTML data
* minor code format fix
* PR Changes
NOTE: refactoring text buffer code is a separate task. New issue to be created.
* Refactor TextBuffer::GenHTML (#2038)
Fixes#1846.
* nit change
* x86 build fix
* nit changes
…. It will return a replacement character at that point if it was given bad data. #788
<!-- Enter a brief description/summary of your PR here. What does it fix/what does it change/how was it tested (even manually, if necessary)? -->
## Summary of the Pull Request
This modifies the parser used while inserting text into the underlying data buffer to never return an empty sequence. The empty sequence is invalid as you can't insert a "nothing" into the buffer. The buffer asserted this with a fail fast crash. Now we will instead insert U+FFFD (the Unicode replacement character) � to symbolize that something was invalid and has been replaced.
<!-- Please review the items on the PR checklist before submitting-->
## PR Checklist
* [x] Closes#788 and internal MSFT: 20990158
* [x] CLA signed. If not, go over [here](https://cla.opensource.microsoft.com/microsoft/Terminal) and sign the CLA
* [x] Tests added/passed
* [x] Requires documentation to be updated
* [x] I've discussed this with core contributors already. If not checked, I'm ready to accept this work might be rejected in favor of a different grand plan. Issue number where discussion took place: #788
<!-- Provide a more detailed description of the PR, other things fixed or any additional comments/features here -->
## Detailed Description of the Pull Request / Additional comments
The solution here isn't perfect and isn't going to solve all of our problems. I was basically trying to stop the crash while not getting in the way of the other things coming down the pipe for the input channels.
I considered the following:
1. Remove the fail fast assertion from the buffer
- I didn't want to do this because it really is invalid to get all the way to placing the text down into the buffer and then request a string of 0 length get inserted. I feel the fail fast is a good indication that something is terribly wrong elsewhere that should be corrected.
2. Update the UTF16 parser in order to stop returning empty strings
- This is what I ultimately did. If it would ever return just a lead, it returns �. If it would ever return just a trail, it returns �. Otherwise it will return them as a pair if they're both there, or it will return a single valid codepoint. I am now assuming that if the parse function is being called in an Output Iterator and doesn't contain a string with all pieces of the data that are needed, that someone at a higher level messed up the data, it is in valid, and it should be repaired into replacements.
- This then will move the philosophy up out of the buffer layer to make folks inserting into the buffer identify half a sequence (if they're sitting on a stream where this circumstance could happen... one `wchar_t` at a time) and hold onto it until the next bit arrives. This is because there can be many different routes into the buffer from many different streams/channels. So buffering it low, right near the insertion point, is bad as it might pair loose `wchar_t` across stream entrypoints.
3. Update the iterator, on creating views, to disallow/transform empty strings.
- I considered this solution as well, but it would have required, under some circumstances, a second parsing of the string to identify lead/trail status from outside the `Utf16Parser` class to realize when to use the � character. So I avoided the double-parse.
4. Change the cooked read classes to identify that they pulled the lead `wchar_t` from a sequence then try to pull another one.
- I was going to attempt this, but @adiviness said that he tried it and it made all sorts of other weirdness happen with the edit line.
- Additionally, @adiviness has an outstanding series of effort to make cooked read significantly less horrible and disgusting. I didn't want to get in the way here.
5. Change the `GetChar` method off of the input buffer queue to return a `char32_t`, a `wstring_view`, transform a standalone lead/trail, etc.
- The `GetChar` method is used by several different accessors and API calls to retrieve information off of the input queue, transforming the Key events into straight up characters. To change this at that level would change them all. Long-term, it is probably warranted to do so as all of those consumers likely need to become aware of handling UTF-16 surrogates before we can declare victory. But two problems.
1. This gets in the way of @adiviness work on cooked read data
2. This goes WAY beyond the scope of what I want to accomplish here as the immediate goal is to stop the crash, not fix the world.
I've validated this by:
1. Writing some additional tests against the Utf16Parser to simulate some of the theoretical sequences that could arrive and need to be corrected into replacement characters per a verbal discussion and whiteboarding with @adiviness.
2. Manually triggered the emoji panel and inserted a bunch of emoji. Then seeked around left and right, deleted assorted points with the backspace key, pressed enter to commit, and used the up-arrow history to recommit them to see what happened. There were no crashes. The behavior is still weird and not great... but outside the scope of no crashy crashy.
* static analysis fixes
* using C++ style casts
* explicit delete changed to reset(nullptr)
* fix for null apiMsg.OtherId during tracing in Compare()
* changed INVALID_ID macro to constexpr
* properly handle null ReplyMsg in ConsoleIoThread()
* Fixed wrong static_cast for State.InputBuffer
* compensate for null reply message to fix deref problem of ReplyMsg in srvinit.cpp by changing signature in DeviceComm.h
