Logo
About usInnovation CMChallengesEuropa2iEntrepreneurshipR&D&I SearchAgentsEventsReports
en
METHOD AND DEVICE FOR SYNCHRONISING SUBTITLES WITH AUDIO FOR LIVE SUBTITLINGCM Patents

Índice de la ficha

Updated at
24/07/2026
Numero publicacion
EP.2574054.A1
Fecha publicacion
27/03/2013
Numero solicitud
EP20110783108
Fecha presentacion
20/05/2011

En detalle

Resumen

[0001] The present invention relates to a method and device for synchronizing subtitles with audio in live subtitling, wherein live subtitle and audio synchronization that is proposed determines an individual subtitle delay Δt<i>of each subtitle i and creates a subtitle unit S<i>with the subtitle i and the individual subtitle delay Δt<i>. Each subtitle unit S<i>is introduced in one or more subtitle PES packets with a presentation time stamp PTS<i>and together with a configurable associated shift delay REST. Said packets are part of an extended transport stream including audio packets containing at least one audio fragment j and a presentation time stamp PTS<j>. The synchronization of subtitle packets with audio packets is performed by creating a synchronized transport stream comprising all the packets of the extended transport stream delayed a time depending on a configurable associated shift delay REST.

Reivindicaciones

1. Method for synchronizing subtitles with audio in live subtitling systems, which comprises: - generating at least one subtitle i from at least one transcription block T<j> corresponding to an audio fragment j; characterized in that it further comprises: - determining an individual subtitle delay Δt<i> of each subtitle i; - creating a subtitle unit S<i> comprising the subtitle i and the individual subtitle delay Δt<i> ; - putting each subtitle unit S<i> together with a configurable associated shift delay REST in at least one subtitle PES packet, which has a presentation time stamp PTS<i> , said subtitle PES packets being part of an extended transport stream including at least one audio packet containing the audio fragment j and a presentation time stamp PTS<j> . - synchronizing subtitle PES packets with audio packets by creating a synchronized transport stream comprising all the packets of the extended transport stream delayed a time depending on the configurable associated shift delay REST. 2. Method according to claim 1, wherein the synchronization of subtitle PES packets with audio packets comprises delaying each subtitle PES packet by associating it with a presentation time stamp in the synchronized transport stream which is equal to PTS<i> -Δt<i> +REST, and delaying each audio packet by associating it with a presentation time stamp in the synchronized transport stream which is equal to PTS<j> +REST, PTS<i> and PTS<j> being the presentation time stamps in the extended transport stream and REST being the associated shift delay of the extended transport stream. 3. Method according to claim 2, wherein if the extended transport stream includes at least one video packet or at least one data packet of any class of data synchronizable with audio packets, the synchronization of subtitle packets further comprises delaying all the video and data packets a time depending on the configurable associated shift delay REST. 4. Method according to any of claims 1 to 3, wherein the synchronization of subtitle packets with audio packets is performed before transmitting the synchronized transport stream to a digital receiving device. 5. Method according to claim 4, which comprises receiving the synchronized transport stream in the digital receiving device, which is a conventional receiver selected from a digital radio broadcasting receiver, a digital terrestrial television decoding module, an IP television receiver, a digital multimedia broadcasting receiver and a digital media receiver with control means for live television. 6. Method according to any of claims 1 to 3, wherein the synchronization of subtitle packets with audio packets is performed after receiving the extended transport stream in a digital receiving device. 7. Method according to claim 6, which further comprises allowing a user to choose whether or not the synchronization of subtitle packets with audio packets is performed by the digital receiving device. 8. Method according to any of claims 6 to 7, which comprises receiving the extended transport stream in the digital receiving device, which is selected from a digital radio broadcasting receiver, a digital terrestrial television decoding module, an IP television receiver, a digital multimedia broadcasting receiver and a digital media receiver with control means for live television. 9. Method according to any of claims 1 to 3, wherein the synchronization of subtitle packets with audio packets is performed before storing the synchronized transport stream in a multimedia storage medium. 10. Method according to any of claims 1 to 3, wherein the synchronization of subtitle packets with audio packets is performed after storing the extended transport stream in a multimedia storage medium and before transmitting the synchronized transport stream to a multimedia distribution network, which is selected from an IP network, a digital terrestrial television broadcasting network and a digital radio broadcasting network. 11. Method according to any preceding claim, wherein the determination of an individual subtitle delay Δt<i> of each subtitle i comprises: - determining a beginning time stamp t<jB> which is the moment in which the first audio fragment corresponding to the first transcription block T<j> of a group of transcription blocks begins, the group formed by at least one transcription block and created by an audio transcription sub-system 201, and from which at least one subtitle unit S<i> is generated by means of a subtitle generation sub-system 202, with respect to a reference clock 100, the reference clock 100 being used by both the subtitle generation sub-system 202 and by the audio transcription sub-system 201; - determining a delivery time tsub<i> which is when the first subtitle generated from said group of transcription blocks by the subtitle generation sub-system 202 is delivered; - subtracting the beginning time stamp t<jB> of said first audio fragment from the delivery time tsub<i> to calculate the individual subtitle delay Δt<i> = tsub<i> - t<jB> ; - assigning said individual subtitle delay Δt<i> to each subtitle unit S<i> created from said group of transcription blocks. 12. Method according to claim 11, which further comprises determining an end time stamp t<jE> which is the moment in which the first audio fragment corresponding to the first transcription block T<j> of the group of transcription blocks ends with respect to the reference clock 100. 13. Method according to claim 12, wherein the audio transcription sub-system 201 uses a mental transcription by a person and the beginning time stamp t<jB> and the end time stamp t<jE> are estimated respectively as t<jB> = th<jB> + Δt<mpj> and t<jE> = th<jE> + Δt<mpj> by calculating a mental processing delay Δt<mpj> depending on the type of mental transcription, which is selected from continuous mental transcription and discrete mental transcription, th<jB> is a stamped instant in time when the person begins the mental transcription and th<jE> is a stamped instant in time when the person ends the mental transcription, both times given automatically by the audio transcription sub-system 201 with respect to the reference clock 100. 14. Method according to claim 13, wherein the mental processing delay Δt<mpj> is a constant established by the audio transcription sub-system 201 when it uses continuous mental transcription. 15. Method according to claim 13, wherein the mental processing delay Δt<mpj> is calculated by the audio transcription sub-system 201 when it uses discrete mental transcription as Δt<mpj> = TextLength<j> · (K/µ) + C, TextLength<j> being a length in units of length of the audio fragment j, K is the time duration of a unit of length; C is a time constant configurable in the audio transcription sub-system 201, and µ is a degree of literality variable, 0≤µ≤1, which is configurable and represents the proximity of the mental transcription to literality. 16. Subtitle synchronizing device, characterized in that it comprises processing means configured for: - creating an extended transport stream from an initial transport stream comprising all the packets of the initial transport stream, which have an associated presentation time stamp, the extended transport stream comprising: at least one audio packet containing an audio fragment j, at least one subtitle unit S<i> comprising a subtitle i generated from at least one transcription block T<j> corresponding to the audio fragment j and a specific individual subtitle delay Δt<i> associated with each subtitle i, and an associated shift delay REST which is configurable and indicates an established delay for synchronizing subtitle packets with audio packets, - adjusting the associated presentation time stamp of each subtitle packet of the extended transport stream by associating it with a presentation time stamp in the synchronized transport stream which is equal to PTS<i> -Δt<i> +REST, and delaying each audio packet by associating it with a presentation time stamp in the synchronized transport stream which is equal to PTS<j> +REST, PTS<i> and PTS<j> being presentation time stamps associated with the subtitle packet and the audio packet, respectively, in the extended transport stream and REST being the associated shift delay in the extended transport stream. 17. Subtitle synchronizing device according to claim 16, wherein if the initial transport stream includes at least one video packet or at least one data packet of any class of data synchronizable with audio packets, the processing means are configured for delaying all the video and data packets the global shift time. 18. Subtitle synchronizing device according to any of claims 16 to 17, wherein the processing means are configured for creating the synchronized transport stream before transmitting said stream to a digital receiving device. 19. Subtitle synchronizing device according to any of claims 16 to 17, wherein the processing means are configured for allowing a user to choose if the creation of the synchronized transport stream is performed after receiving the extended transport stream in a digital receiving device. 20. Subtitle synchronizing device according to claim 19, which is integrated in the digital receiving device selected from a digital radio broadcasting receiver, a digital terrestrial television decoding module, an IP television receiver, a digital multimedia broadcasting receiver and a digital media receiver with control means for live television. 21. Subtitle synchronizing device according to any of claims 16 to 17, wherein the processing means are configured for creating the synchronized transport stream before storing said stream in a multimedia storage medium. 22. Subtitle synchronizing device according to any of claims 16 to 17, wherein the processing means are configured for creating the synchronized transport stream from an extended transport stream previously stored in a multimedia storage medium and before transmitting the synchronized transport stream to a multimedia distribution network, which is selected from an IP network, a digital terrestrial television broadcasting network and a digital radio broadcasting network. 23. Software product comprising program code means which when loaded in a general purpose processor, a digital signal processor, application-specific integrated circuits or any programmable logic device, makes said program code means execute the method according to any of claims 1 to 15.

Etiquetas

Inventores
De Castro Alvarez MercedesDe Pedro Sastre ManuelRuiz Mezcua Maria BelenJimenez Dorado Javier
Solicitantes
Universidad Carlos III de MadridDe Castro Alvarez MercedesDe Pedro Sastre ManuelRuiz Mezcua Maria BelenJimenez Dorado Javier
Clasificacion ipc
H04N 21/ 2187 A IH04N 21/ 2343 A IH04N 21/ 43 A IH04N 21/ 488 A IH04N 7/ 24 A I
Logo

Innovation CM
Challenges
Europa2i
Entrepreneurship
R&D&I Search
Agents
Events
Reports
About us
Contact
Give us your opinion
Cookies
Legal notice
Privacy

© Copyright Espacio Madrileño de Investigación e Innovación 2026