for issue #4418, #4151, #4076 .DVR Missing First Few Seconds of
Audio/Video
### Root Cause
When recording WebRTC streams to FLV files using DVR, the first 4-6
seconds of audio/video are missing. This occurs because:
1. **Packets are discarded before A/V sync is available**: The
RTC-to-RTMP conversion pipeline actively discards all RTP packets when
avsync_time <= 0.
2. **Original algorithm requires 2 RTCP SR packets**: The previous
implementation needed to receive two RTCP Sender Report (SR) packets
before it could calculate the rate for audio/video synchronization
timestamp conversion.
3. **Delay causes packet loss**: Since RTCP SR packets typically arrive
every 2-3 seconds, waiting for 2 SRs means 4-6 seconds of packets are
discarded before A/V sync becomes available.
4. **Audio SR arrives slower than video SR**: As reported in the issue,
video RTCP SR packets arrive much faster than audio SR packets. This
asymmetry causes audio packets to be discarded for a longer period,
resulting in the audio loss observed in DVR recordings.
### Solution
1. **Initialize rate from SDP**: Use the sample rate from SDP (Session
Description Protocol) to calculate the initial rate immediately when the
track is created.
Audio (Opus): 48000 Hz → rate = 48 (RTP units per millisecond)
Video (H.264/H.265): 90000 Hz → rate = 90 (RTP units per millisecond)
2. **Enable immediate A/V sync:** With the SDP rate available,
cal_avsync_time() can calculate valid timestamps from the very first RTP
packet, eliminating packet loss.
3. **Smooth transition to precise rate**: After receiving the 2nd RTCP
SR, update to the precisely calculated rate based on actual RTP/NTP
timestamp mapping.
## Configuration
Added new configuration option `init_rate_from_sdp` in the RTC vhost
section:
```nginx
vhost rtc.vhost.srs.com {
rtc {
# Whether initialize RTP rate from SDP sample rate for immediate A/V sync.
# When enabled, the RTP rate (units per millisecond) is initialized from the SDP
# sample rate (e.g., 90 for video 90kHz, 48 for audio 48kHz) before receiving
# 2 RTCP SR packets. This allows immediate audio/video synchronization.
# The rate will be updated to a more precise value after receiving the 2nd SR.
# Overwrite by env SRS_VHOST_RTC_INIT_RATE_FROM_SDP for all vhosts.
# Default: off
init_rate_from_sdp off;
}
}
```
**⚠️ Important Note**: This config defaults to **off** because:
- ✅ When **enabled**: Fixes the audio loss problem (no missing first 4-6
seconds)
- ❌ When **enabled**: VLC on macOS cannot play the video properly
- ✅ Other platforms work fine (Windows, Linux)
- ✅ FFplay works fine on all platforms
Users experiencing audio loss in DVR recordings can enable this option
if they don't need VLC macOS compatibility. We're investigating the VLC
macOS issue to make this feature safe to enable by default in the
future.
---------
Co-authored-by: winlin <winlinvip@gmail.com>
Co-authored-by: OSSRS-AI <winlinam@gmail.com>
This PR introduces anonymous coroutine macros for easier coroutine
creation and improves the State Threads (ST) mutex and condition
variable handling in SRS.
- **Added coroutine macros**: `SRS_COROUTINE_GO`, `SRS_COROUTINE_GO2`,
`SRS_COROUTINE_GO_CTX`, `SRS_COROUTINE_GO_CTX2`
- **Added `SrsCoroutineChan`**: Channel for sharing data between
coroutines with coroutine-safe operations
- **Simplified coroutine creation**: Go-like syntax for creating
anonymous coroutines with code blocks
---------
Co-authored-by: Jacob Su <suzp1984@gmail.com>
Co-authored-by: OSSRS-AI <winlinam@gmail.com>
Fixes a bug in WebRTC NACK packet recovery mechanism where recovered
packets were being discarded instead of processed.
In `SrsRtcRecvTrack::on_nack()`, when a retransmitted packet arrived
(found in NACK receiver), the method would:
1. ✅ Remove the packet from NACK receiver (correct)
2. ❌ Return early without adding the packet to RTP queue (BUG)
This caused recovered packets to be lost, defeating the purpose of the
NACK mechanism and potentially causing media quality issues.
Restructured the control flow in `on_nack()` to ensure both new and
recovered packets reach the packet insertion logic:
- **Before**: Early return for recovered packets → packets discarded
- **After**: Conditional NACK management + unified packet processing →
all packets queued
Closes#3820
---------
Co-authored-by: Haibo Chen <495810242@qq.com>
Co-authored-by: OSSRS-AI <winlinam@gmail.com>
This PR modernizes SRS's HTTP handling by upgrading from the legacy
http-parser library to the more performant and actively maintained
llhttp library.
* Replace http-parser with llhttp: Migrated from the deprecated
http-parser to llhttp for better performance and maintenance
* API compatibility: Updated all HTTP parsing logic to use llhttp APIs
while maintaining backward compatibility
* Simplified URL parsing: Replaced complex http-parser URL parsing with
custom simple parser implementation
Enhanced error handling: Improved error reporting with llhttp's better
error context and positioning
---------
Co-authored-by: OSSRS-AI <winlinam@gmail.com>
This PR introduces a major refactoring to replace `SrsSharedPtrMessage`
with `SrsMediaPacket` throughout the SRS codebase, providing a more
unified and cleaner approach to media packet handling.
---------
Co-authored-by: OSSRS-AI <winlinam@gmail.com>
This PR consolidates the SRT and RTC server functionality into the main
SrsServer class, eliminating the separate `SrsSrtServer` and
`SrsRtcServer` classes and their corresponding adapter classes. This
architectural change simplifies the codebase by removing the hybrid
server pattern and integrating all protocol handling directly into
`SrsServer`.
As unified connection manager (`_srs_conn_manager`) for all protocol
connections, all incoming connections are checked against the same
connection limit in `on_before_connection()`. This enables consistent
connection limits: `max_connections` now protects against resource
exhaustion from any protocol, not just RTMP.
Remove modules because it's not used now, so only keep the server
application module and main entry point. Remove the wait group to run
server, instead, directly run server and invoke the cycle method.
After this PR, the startup workflow and servers architecture should be
much easier to maintain.
---------
Co-authored-by: OSSRS-AI <winlinam@gmail.com>
This PR introduces a comprehensive stream publish token system that
prevents race conditions when multiple publishers attempt to publish to
the same stream URL simultaneously across different protocols (RTMP,
WebRTC, SRT).
* Race Condition Issue: Multiple publishers could create duplicate
sources for the same stream when context switches occurred during source
initialization in SRS's coroutine-based architecture
* Cross-Protocol Conflicts: Different protocols (RTMP, RTC, SRT) could
simultaneously publish to the same stream URL without coordination
* Resource Management: No centralized mechanism to ensure exclusive
stream publishing access
---------
Co-authored-by: OSSRS-AI <winlinam@gmail.com>
This PR fixes a critical race condition in SRS source managers where
multiple coroutines could create duplicate sources for the same stream.
- **Atomic source creation**: Source lookup, creation, and pool
insertion now happen atomically within lock scope
- **Consistent interface**: Standardize on `ISrsRequest*` interface
throughout codebase
- **Handler simplification**: Remove `ISrsLiveSourceHandler*` parameter,
obtain from global server instance
---------
Co-authored-by: OSSRS-AI <winlinam@gmail.com>
This PR makes WebRTC a core feature of SRS and enforces C++98
compatibility by:
1. Always Enable WebRTC Support
- Remove `--rtc=on|off` configuration option - WebRTC is now always
enabled
- Eliminate all `#ifdef SRS_RTC` conditional compilation blocks
- Include WebRTC-related modules (RTC, SRTP, DTLS) in all builds
- Update build scripts to always link WebRTC dependencies
2. Enforce C++98 Compatibility
- Remove `--cxx11=on|off` and `--cxx14=on|off` configuration options
- Force `SRS_CXX11=NO` and `SRS_CXX14=NO` in build system
- Move these options to deprecated section with warnings
- Ensure codebase maintains C++98 standard compatibility
3. Remove Windows/Cygwin Support
- Remove all Windows and Cygwin64 conditional compilation blocks (#ifdef
_WIN32, #ifdef CYGWIN64)
- Delete Cygwin64 build configurations from build scripts (
auto/options.sh, auto/depends.sh, configure)
- Remove Cygwin64 assembly files and State Threads platform support (
md_cygwin64.S)
- Eliminate Windows-specific GitHub Actions workflows and CI/CD jobs
- Remove NSIS packaging files and Windows installer generation
- Delete Windows documentation and update feature lists to mark support
as removed in v7.0
- Simplify OS detection to only support Unix-like systems (Linux, macOS)
4. Code Cleanup
- Remove conditional WebRTC code blocks throughout the codebase
- Simplify build configuration by removing WebRTC-related conditionals
- Update constructor delegation patterns to be C++98 compatible
- Fix vector initialization to use C++98 syntax
- Eliminate Windows-specific implementations for file operations, time
handling, and networking
- Unified platform handling with consistent POSIX API usage
---------
Co-authored-by: OSSRS-AI <winlinam@gmail.com>
This PR removes the multi-threading infrastructure from SRS and
consolidates the codebase to use single-thread architecture exclusively.
This is a architectural simplification that aligns with SRS's
coroutine-based design philosophy.
* Simplified Architecture: Eliminates complexity of multi-threading
coordination
* Better Alignment: Matches SRS's coroutine-based single-thread design
philosophy
* Reduced Complexity: Removes potential race conditions and threading
bugs
* Cleaner Code: More focused modules with clear responsibilities
* Easier Maintenance: Fewer moving parts and clearer execution flow
---------
Co-authored-by: OSSRS-AI <winlinam@gmail.com>
Currently, SRS only supports HLS with MPEG-TS format segment files, but
for LL-HLS and HEVC, it requires the fMP4 format. See #4327 for details.
Furthermore, fMP4 has a smaller overhead compared to TS, and fMP4 can be
used for DVR. In short, fMP4 is definitely the future segment format for
HLS.
Start SRS with the config file that enables HLS with fMP4:
```
./objs/srs -c conf/hls.mp4.conf
```
Publish stream by FFmpeg:
```
ffmpeg -re -i doc/source.flv -c copy -f flv rtmp://localhost/live/livestream
```
Play the stream by SRS player:
[http://localhost:8080/live/livestream.m3u8](http://localhost:8080/players/srs_player.html?stream=livestream.m3u8)
Finished by AI:
* [AI: Change init.mp4 to the same directory of
m3u8.](17621c8442)
* [AI: Fix the error handling
bug.](af3758a592)
* [AI: Fix Chrome stuttering
problem.](aaab60c314)
---------
Co-authored-by: winlin <winlinvip@gmail.com>
Rtp packets may be retransmitted, disordered, jittery, delayed,
etc.There may be abnormalities when converting to rtmp.
To reproduce this problem, you need to set the network reordering by
[tc-ui](https://github.com/ossrs/tc-ui). Note that you need a linux
server, and start it by docker:
```bash
docker run --network=host --privileged -it --restart always -d \
--name tc -v /lib/modules:/lib/modules:ro ossrs/tc-ui:1
```
Set up 5% packet reordering and a 1ms delay; then you will notice that
the audio is stuttering, somewhat noisy, and lacks fluency.
```bash
curl http://localhost:2023/tc/api/v1/config/raw -X POST \
-d 'tcset ens5 --direction incoming --delay 40ms --reordering 5% --port 8000'
```
> Note: Even without network conditions, the natural state can also
cause packet reordering, especially in public cloud platforms such as
AWS EC2.
> Note: You can use command `curl
http://localhost:2023/tc/api/v1/config/raw -X POST -d 'tcdel --all
ens5'` to reset the network condition settings.
Check the web console, you will see the reordering setup:
<img width="500" alt="TC Settings"
src="https://github.com/user-attachments/assets/b278fdf4-9fcc-4aac-b534-dfa34e28c371"
/>
Then, publish stream via WHIP: http://localhost:8080/players/whip.html
And, play via HTTP-FLV: http://localhost:8080/players/srs_player.html
Finished by AI:
* [AI: Extract audio jitter buffer to class
AudioPacketCache](a4097d9374)
* [AI: Add utest and fix
bug.](c919227af5)
---------
Co-authored-by: Haibo Chen <495810242@qq.com>
Co-authored-by: winlin <winlinvip@gmail.com>
## Introduce
This PR adds support for viewing streams via the RTSP protocol. Note
that it only supports viewing streams, not publishing streams via RTSP.
Currently, only publishing via RTMP is supported, which is then
converted to RTSP. Further work is needed to support publishing RTC/SRT
streams and converting them to RTSP.
## Usage
Build and run SRS with RTSP support:
```
cd srs/trunk && ./configure --rtsp=on && make -j16
./objs/srs -c conf/rtsp.conf
```
Push stream via RTMP by FFmpeg:
```
ffmpeg -re -i doc/source.flv -c copy -f flv rtmp://localhost/live/livestream
```
View the stream via RTSP protocol, try UDP first, then use TCP:
```
ffplay -i rtsp://localhost:8554/live/livestream
```
Or specify the transport protocol with TCP:
```
ffplay -rtsp_transport tcp -i rtsp://localhost:8554/live/livestream
```
## Unit Test
Run utest for RTSP:
```
./configure --utest=on & make utest -j16
./objs/srs_utest
```
## Regression Test
You need to start SRS for regression testing.
```
./objs/srs -c conf/regression-test-for-clion.conf
```
Then run regression tests for RTSP.
```
cd srs/trunk/3rdparty/srs-bench
go test ./srs -mod=vendor -v -count=1 -run=TestRtmpPublish_RtspPlay
```
## Blackbox Test
For blackbox testing, SRS will be started by utest, so there is no need
to start SRS manually.
```
cd srs/trunk/3rdparty/srs-bench
go test ./blackbox -mod=vendor -v -count=1 -run=TestFast_RtmpPublish_RtspPlay_Basic
```
## UDP Transport
As UDP requires port allocation, this PR doesn't support delivering
media stream via UDP transport, so it will fail if you try to use UDP as
transport:
```
ffplay -rtsp_transport udp -i rtsp://localhost:8554/live/livestream
[rtsp @ 0x7fbc99a14880] method SETUP failed: 461 Unsupported Transport
rtsp://localhost:8554/live/livestream: Protocol not supported
[2025-07-05 21:30:52.738][WARN][14916][7d7gf623][35] RTSP: setup failed: code=2057
(RtspTransportNotSupported) : UDP transport not supported, only TCP/interleaved mode is supported
```
There are no plans to support UDP transport for RTSP. In the real world,
UDP is rarely used; the vast majority of RTSP traffic uses TCP.
## Play Before Publish
RTSP supports audio with AAC and OPUS codecs, which is significantly
different from RTMP or WebRTC.
RTSP uses commands to exchange SDP and specify the audio track to play,
unlike WHEP or HTTP-FLV, which use the query string of the URL. RTSP
depends on the player’s behavior, making it very difficult to use and
describe.
Considering the feature that allows playing the stream before publishing
it, it requires generating some default parameters in the SDP. For OPUS,
the sample rate is 48 kHz with 2 channels, while AAC is more complex,
especially regarding the sample rate, which may be 44.1 kHz, 32 kHz, or
48 kHz.
Therefore, for RTSP, we cannot support play-then-publish. Instead, there
must already be a stream when playing it, so that the audio codec is
determined.
## Opus Codec
No Opus codec support for RTSP, because for RTC2RTSP, it always converts
RTC to RTMP frames, then converts them to RTSP packets. Therefore, the
audio codec is always AAC after converting RTC to RTMP.
This means the bridge architecture needs some changes. We need a new
bridge that binds to the target protocol. For example, RTC2RTMP converts
the audio codec, but RTC2RTSP keeps the original audio codec.
Furthermore, the RTC2RTMP bridge should also support bypassing the Opus
codec if we use enhanced-RTMP, which supports the Opus audio codec. I
think it should be configurable to either transcode or bypass the audio
codec. However, this is not relevant to RTSP.
## AI Contributor
Below commits are contributed by AI:
* [AI: Remove support for media transport via
UDP.](755686229f)
* [AI: Add crutial logs for each RTSP
stage.](9c8cbe7bde)
* [AI: Support AAC doec for
RTSP.](7d7cc12bae)
* [AI: Add option --rtsp for
RTSP.](f67414d9ee)
* [AI: Extract SrsRtpVideoBuilder for RTC and
RTSP.](562e76b904)
---------
Co-authored-by: Jacob Su <suzp1984@gmail.com>
Co-authored-by: winlin <winlinvip@gmail.com>
**Introduce**
This pull request builds upon the foundation laid in
https://github.com/ossrs/srs/pull/4289 . While the previous work solely
implemented unidirectional HEVC support from RTMP to RTC, this
submission further enhances it by introducing support for the RTC to
RTMP direction.
**Usage**
Launch SRS with `rtc2rtmp.conf`
```bash
./objs/srs -c conf/rtc2rtmp.conf
```
**Push with WebRTC**
Upgrade browser to Chrome(136+) or Safari(18+), then open [WHIP
encoder](http://localhost:8080/players/whip.html?schema=http&&codec=hevc),
push stream with URL that enables HEVC by query string `codec=hevc`:
```bash
http://localhost:1985/rtc/v1/whip/?app=live&stream=livestream&codec=hevc
```
This query string `codec=hevc` is used to select the video codec, and
generate lines in the answer SDP.
```
m=video 9 UDP/TLS/RTP/SAVPF 49 123
a=rtpmap:49 H265/90000
```
The encoder log also show the codec:
```
Audio: opus, 48000HZ, channels: 2, pt: 111
Video: H265, 90000HZ, pt: 49
```
**Play with RTMP**
Play HEVC stream via RTMP.
```bash
ffplay -i rtmp://localhost/live/livestream
```
You will see the codec in logs:
```
Stream #0:0: Audio: aac (LC), 48000 Hz, stereo, fltp
Stream #0:1: Video: hevc (Main), yuv420p(tv, bt709), 320x240, 30 fps, 30 tbr, 1k tbn
```
You can also use [WHEP
player](http://localhost:8080/players/whep.html?schema=http&&codec=hevc)
to play the stream.
Important refactor with AI:
* [AI: Refactor packet cache for RTC frame
builder.](b8ffa1630e)
* [AI: Refactor the packet copy and free for
SrsRtcFrameBuilder](f3487b45d7)
* [AI: Refactor the frame detector for
SrsRtcFrameBuilder](4ffc1526b9)
* [AI: Refactor the packet_video_rtmp for
SrsRtcFrameBuilder](81f6aef4ed)
* [AI: Add utests for
SrsCodecPayload.codec](61eb1c0bfc)
* [AI: Add utests for VideoPacketCache in
SrsRtcFrameBuilder.](fd25480dfa)
* [AI: Add utests for VideoFrameDetector in
SrsRtcFrameBuilder.](b4aa977bbd)
* [AI: Add regression test for RTC2RTMP with
HEVC.](5259a2aac3)
---------
Co-authored-by: Jacob Su <suzp1984@gmail.com>
Co-authored-by: winlin <winlinvip@gmail.com>
