To begin with, Kant himself did not speak about courtesy of the heart [Höflichkeit des Herzens] in any of his texts. I made a full-text search in German on the Academy Edition over all 23 volumes - including all written works, lecture notes (also of his students), letters, and fragments and notes - with "des Herzens" (of the heart). The term did not pop up in any of the main works, but was included in some of the fragments, letters and less known works (like the one reflecting on medicine). But never in connection with courtesy. I went though every single hit.
The two text bits that could clarify Kant's own stance towards "of the heart" in specifically moral context I could find are the following:
Reflection 1179, Ak. 15:521.24-7
Die moralitaet besteht keinesweges in der gutartigkeit des Herzens, sondern in dem guten Charakter, und den soll sie bilden.
Die Anpreisungen des Guten Herzens sind eine rechte Nahrung vor die Eigenliebe, ...
Morality in no way consists in benignancy of the heart, but in good character, and that is what it should form.
The extolments of the good heart are good nurturing of self-love,...
I think that it makes clear that dignity, essentially linked to morality, can hardly be seen as "courtesy of the heart".
Reflection 6858, Ak. 19:181.20-22
Die Glükseeligkeit ist in diesem Leben nicht ihre Aufmunterung; überdem ist die reine Gesinnung des Herzens das, was den eigentlichen moralischen werth ausmacht; ...
The happiness is, in this life, not their [i.e. moral imperatives] incentive; furthermore, it is the pure attitude of the heart that constitutes its real moral worth; ...
Here, pure attitude of the heart is essentially the same thing as what Kant calls "character". Actually character is directly linked with moral thinking as well as dignity (most explicitly in the Anthropology, Ak. 7: 291-5).
Kant does speak about Güte, i.e. goodness or benevolence, of the heart more often, but this is pretty much the same as as it is in the first reflection and the same objections are raised.
Regarding the roots of "courtesy of heart"
"Courtesy of the heart" is a phrase used by Goethe in Die Wahlverwandtschaften (1809, chapter 24):
Es gibt eine Höflichkeit des Herzens; sie ist der Liebe verwandt. Aus ihr entspringt die bequemste Höflichkeit des äußern Betragens.
There is a courtesy of the heart; it is akin to love. Out of it arises the purest courtesy in the outward behaviour.
This whole section is the work of @ChristropheStrobbe in form of a comment under the question and included by me for completeness. A very good find indeed!
Regarding the link of this term to Kant
Especially considering that both authors link that phrase to Kant's last days, I think the most probable original source is Ernst Cassirer's classic Kant's Life and Thought, originally published in German in 1918 (see Kants Leben und Lehre in the Internet Archive - Once more merits to @ChristopheStrobbe for looking up and editing in that reference).
There, he writes:
His [Kant's] relationship to individuals was guided and ruled by universal respect for the freedom of the moral person and his right of self-determination. And this
respect was no abstract demand, but it acted on him as an immediately
living motive, determining every particular utterance. By
this disposition Kant acquired that "courtesy of the heart" which, if
not precisely the same as love, is nonetheless related to love. His
"sense of humanity," which he held fast to and guarded until the last
days of his life, was divorced from every merely sentimental subsoil. (Translation by James Haden, 1981, p. 414, emphasis mine)
All this is embedded in descriptions of Kant's personality and moral integrity by contemporaries. I think what we can see here is threefold: 1) both phrases are used 2) Cassirer tries to describe Kant's mindset and 3) there is no reference to Kant's own writing, which Cassirer would have done if there were any, considering the academic integrity he had and his usual methodology.
Interestingly, Cassirer uses this term to describe the character of Kant's behaviour as explicitely not 'loving' and 'emotionally warm', but nevertheless something similar and much more pure, just as Goethe did.
Just to be clear: The story with the doctor and the "sense of humanity" that had not left Kant is a transcript of oral utterances of Kant, testified by Wasianski from memory, see Cassirer (1918:439-440;1981:412-413). He refers to this story from two pages earlier here. The "courtesy of the heart" [Höflichkeit des Herzens] in contrast clearly is a descriptive term used Cassirer himself, as is explicitly pointed out in the introduction to the English edition, see Cassirer (1981:xxi). And it is probably taken from Goethe, not Kant!
Sullivan even explicitely refers to Cassirer (p. 412-13, 414) in the footnote no. 6 at the end of the paragraph (looked it up in the library). Cave essentially does not refer in anything he writes in this book to any particular text, but it is reasonable to assume the same thing happened.
There are six points to consider:
- The wording is hardly reconcilable with Kant's writings
- The translation of Cassirer from 1981 is the first mentioning of this term in English linked to Kant I could find (lurking through Google Scholar and the databases the University of Kent has access to for some time)
- Cassirer would have referenced it if there was any reference, as he did with "sense of humanity", although not even to Kant directly.
- The editors of the English edition assumed it was Cassirer's own term in the introduction
- Any other reference to this term I could find, like the two in the OP, is later than 1981
- Sullivan only refers to Cassirer, not to Kant directly, in his footnote
Considering all this, I am convinced that this term is wrongly (or at the very least misleadingly) referred to as Kant's own; in ignorance of Kant's writing, Cassirer's own wording, and the pretty explicit introduction to the English edition. It actually is a term used by Cassirer and probably, considering that he writes about love as well, taken from Goethe. Cassirer used this trying to describe the reality of Kant's own personal moral mindset and behaviour.
This perpetuating of wrong conclusions, as shown by the adaption of this term as Kant's own by various writers, clearly is an example of poor academia.