[mythtv] Mac OS X video: QuickTime implementation
Jeremiah Morris
jm at whpress.com
Mon Sep 6 22:55:01 EDT 2004
Following Nigel's suggestion in videoout_quartz.cpp, I have made some
progress in using QuickTime instead of libavcodec to do YUV->RGB
conversion. (Special thanks to Darrell Walisser for libSDL code that
set me in the right direction.) The attached files probably aren't yet
CVS-ready, but I figured I'd throw it out to Nigel and the group.
New features:
- QuickTime does the YUV conversion and blitting to the screen.
- Can optionally change the display resolution (controlled by a #define
at compile time).
- It uses QuickTime to scale the image, either to fullscreen or to the
GUI window width. Scaling seems to have low overhead, so it's enabled
by default.
Known problems:
- On my PowerBook G4 800, I have frequent "prebuffering pause" and "A/V
diverged" problems. Under v1, I couldn't draw enough frames to trigger
the prebuffering error, so I don't know if I did something wrong.
Changing the kPrebufferFrames number alters, but does not cure, the
problem. This is really where I'm stuck; Nigel, does your superior
hardware fix the issue?
- On my box at least, I have to pass a non-intuitive data offset value
to the decompressor to avoid artifacts. I'm skipping 552 bytes in the
YUV buffer (my frame size is 640x480). Is there some data header that
I'm not understanding in the VideoFrame buffer? At any rate, if your
video looks funny, it's probably this value (line 545).
- There are transparency problems with the OSD overlays - I get yellow
instead of transparent areas in both v1 and this version.
- The over/underscan option reacts funny with the OSD overlays; they
get drawn too small, unless my scaling is off. I haven't tried it out
on a Linux frontend to see if it's a general issue.
What do you think?
- Jeremiah
-------------- next part --------------
/********************************************************************************
* = NAME
* videoout_quartz.cpp
*
* = DESCRIPTION
* Basic video for Mac OS X, using an unholy amalgamation of QuickTime,
* QuickDraw, and Quartz/Core Graphics.
*
* = PERFORMANCE
* Seems to be better than version 1 -- numbers, Nigel?
*
* = KNOWN BUGS
* - Changing video resolution is only controllable at compile time
* - Video always scales to fullscreen or GUI size; to avoid scaling, set
* monitor resolution or GUI size (with "Use GUI size for TV playback")
* to an appropriate value
* - Haven't tested "Live preview" function
* - Doesn't use over/underscan or offset values for playback
* - Uses "magic" offset value for reading pixel data
*
* = REVISION
* $Id: videoout_quartz.cpp,v 1.1 2004/08/19 05:20:10 ijr Exp $
*
* = AUTHORS
* Nigel Pearson, Jeremiah Morris
*******************************************************************************/
// *****************************************************************************
// Configuration:
// Define this if you want to change the display's mode to one
// that more closely matches the resolution of the video stream.
// This may improve performance when scaling (less bytes to copy/scale to))
#define CHANGE_SCREEN_MODE
// Define this if we want to scale each frame to either the fullscreen,
// or the correct aspect ratio. The alternative is to just
// copy the image rectangle to the middle of the screen
#define SCALE_VIDEO
// Default numbers of buffers from some of the other videoout modules:
const int kNumBuffers = 31;
const int kNeedFreeFrames = 1;
const int kPrebufferFrames = 12;
const int kKeepPrebuffer = 2;
// *****************************************************************************
#include <map>
#include <iostream>
using namespace std;
#include "mythcontext.h"
#include "filtermanager.h"
#include "videoout_quartz.h"
#import <CoreGraphics/CGBase.h>
#import <CoreGraphics/CGDisplayConfiguration.h>
#import <CoreGraphics/CGImage.h>
#import <Carbon/Carbon.h>
#import <QuickTime/QuickTime.h>
struct QuartzData
{
// Stored information about the media stream:
int srcWidth,
srcHeight;
float srcAspect;
// What size/position does the user want the stream to be displayed at?
int desiredWidth,
desiredHeight,
desiredXoff,
desiredYoff;
// Information about the display and viewport:
CGDirectDisplayID theDisplay;
bool capturedDisplay; // true if we captured the display
CGrafPtr thePort;
bool drawInWindow;
bool changeResolution;
CFDictionaryRef originalMode,
newMode;
bool capturedBeforeEmbed; // true if capturedDisplay
// mode was used before
// embedding was called
// Structures that we use for decompression:
ImageSequence seqID; // codec sequence identifier
PlanarPixmapInfoYUV420 *pixmap; // frame header + data
size_t pixmapSize; // pixmap size
void * pixelData; // start of data section
size_t pixelSize; // data size
};
VideoOutputQuartz::VideoOutputQuartz(void)
: VideoOutput()
{
Started = 0;
pauseFrame.buf = NULL;
data = new QuartzData();
bzero(data, sizeof(QuartzData));
#ifdef CHANGE_SCREEN_MODE
data->changeResolution = true;
#endif
if (gContext->GetNumSetting("GuiSizeForTV", 0))
{
// If this setting is on, we refrain from full screen
data->drawInWindow = true;
}
}
VideoOutputQuartz::~VideoOutputQuartz()
{
EndDisplay();
if (pauseFrame.buf)
delete [] pauseFrame.buf;
Exit();
delete data;
}
void VideoOutputQuartz::Exit(void)
{
if (Started)
{
Started = false;
DeleteQuartzBuffers();
}
}
/* Tear down vbuffers
*/
void VideoOutputQuartz::DeleteQuartzBuffers()
{
for (int i = 0; i < numbuffers + 1; i++)
{
delete [] vbuffers[i].buf;
vbuffers[i].buf = NULL;
}
}
/* Tear down display changes
*/
void VideoOutputQuartz::EndDisplay(void)
{
// make sure decompression stops first
EndCodec();
if (data->capturedDisplay)
{
DisposePort(data->thePort);
data->thePort = NULL;
if (data->originalMode)
{
CGDisplaySwitchToMode(data->theDisplay, data->originalMode);
data->originalMode = NULL;
}
CGDisplayRelease(data->theDisplay);
data->capturedDisplay = false;
}
}
/* Tear down QuickTime decompressor and buffer storage
*/
void VideoOutputQuartz::EndCodec(void)
{
if (data->seqID)
{
CDSequenceEnd(data->seqID);
data->seqID = NULL;
}
if (data->pixmap)
{
delete [] data->pixmap;
data->pixmap = NULL;
}
}
/* Set the transformation matrix for moving and resizing
* video into our viewport.
*/
void VideoOutputQuartz::Transform(void)
{
if (!data->seqID)
return;
MatrixRecord matrix;
SetIdentityMatrix(&matrix);
int x, y, w, h, sw, sh;
x = data->desiredXoff;
y = data->desiredYoff;
w = data->desiredWidth;
h = data->desiredHeight;
sw = data->srcWidth;
sh = data->srcHeight;
VERBOSE(VB_PLAYBACK, QString("Viewport is %1 x %2").arg(w).arg(h));
VERBOSE(VB_PLAYBACK, QString("Image is %1 x %2").arg(sw).arg(sh));
// constants for transformation operations
Fixed one, zero;
one = Long2Fix(1);
zero = Long2Fix(0);
#ifdef SCALE_VIDEO
// scale width for non-square pixels
if (fabsf(data->srcAspect - (sw * 1.0 / sh)) > 0.01)
{
double aspectScale = data->srcAspect * sh / sw;
VERBOSE(VB_PLAYBACK, QString("Scaling to %1 of width").arg(aspectScale));
ScaleMatrix(&matrix,
X2Fix(aspectScale),
one,
zero, zero);
// reset sw to be apparent width
sw = (int)lroundf(sh * data->srcAspect);
}
// scale to fill viewport
if ((h != sh) || (w != sw))
{
double scale = fmin(h * 1.0 / sh, w * 1.0 / sw);
VERBOSE(VB_PLAYBACK, QString("Scaling to %1 of original").arg(scale));
Fixed scaleFix = X2Fix(scale);
ScaleMatrix(&matrix,
scaleFix, scaleFix,
zero, zero);
// reset sw, sh for new apparent width/height
sw = (int)(sw * scale);
sh = (int)(sh * scale);
}
#endif
// center image in viewport
if ((h != sh) || (w != sw))
{
VERBOSE(VB_PLAYBACK, QString("Centering with %1, %2").arg((w - sw)/2.0).arg((h - sh)/2.0));
TranslateMatrix(&matrix,
X2Fix((w - sw) / 2.0),
X2Fix((h - sh) / 2.0));
}
#ifdef SCALE_VIDEO
// apply over/underscan
int hscan = gContext->GetNumSetting("HorizScanPercentage", 5);
int vscan = gContext->GetNumSetting("VertScanPercentage", 5);
if (hscan || vscan)
{
QString HorizScanMode = gContext->GetSetting("HorizScanMode", "overscan");
QString VertScanMode = gContext->GetSetting("VertScanMode", "overscan");
if (VertScanMode == "underscan")
{
vscan = 0 - vscan;
}
if (HorizScanMode == "underscan")
{
hscan = 0 - hscan;
}
VERBOSE(VB_PLAYBACK, QString("Overscanning to %1, %2").arg(hscan).arg(vscan));
ScaleMatrix(&matrix,
X2Fix((double)(1.0 + (hscan / 100.0))),
X2Fix((double)(1.0 + (vscan / 100.0))),
X2Fix(sw / 2.0),
X2Fix(sh / 2.0));
}
#endif
// apply TV mode offset
int tv_xoff = gContext->GetNumSetting("xScanDisplacement", 0);
int tv_yoff = gContext->GetNumSetting("yScanDisplacement", 0);
if (!embedding && (tv_xoff || tv_yoff))
{
VERBOSE(VB_PLAYBACK, QString("TV offset by %1, %2").arg(tv_xoff).arg(tv_yoff));
TranslateMatrix(&matrix,
Long2Fix(tv_xoff),
Long2Fix(tv_yoff));
}
// apply graphics port or embedding offset
if (x || y)
{
VERBOSE(VB_PLAYBACK, QString("Translating to %1, %2").arg((w - sw)/2.0).arg((h - sh)/2.0));
TranslateMatrix(&matrix,
Long2Fix(x),
Long2Fix(y));
}
// apply matrix to decompressor
SetDSequenceMatrix(data->seqID, &matrix);
}
/* Set the clipping region for only drawing into
* part of the graphics port. This is used for
* the video preview, for instance.
*/
void VideoOutputQuartz::Mask(int x, int y, int w, int h)
{
if (!data->thePort)
return;
if (!data->seqID)
return;
RgnHandle clipRgn = NULL;
Rect portRect;
GetPortBounds(data->thePort, &portRect);
if (!x && !y && !w && !h)
{
// set up desired size based on port
data->desiredXoff = portRect.left;
data->desiredYoff = portRect.top;
data->desiredWidth = (portRect.right - portRect.left);
data->desiredHeight = (portRect.bottom - portRect.top);
}
else
{
// correct offset based on any port coordinate transforms
data->desiredXoff = x + portRect.left;
data->desiredYoff = y + portRect.top;
data->desiredWidth = w;
data->desiredHeight = h;
if ((data->desiredXoff != portRect.left) ||
(data->desiredYoff != portRect.top) ||
(data->desiredWidth != (portRect.right - portRect.left)) ||
(data->desiredHeight != (portRect.bottom - portRect.top)))
{
clipRgn = NewRgn();
OpenRgn();
InsetRgn(clipRgn, data->desiredWidth, data->desiredHeight);
OffsetRgn(clipRgn, data->desiredXoff, data->desiredYoff);
CloseRgn(clipRgn);
}
}
SetDSequenceMask(data->seqID, clipRgn);
if (clipRgn)
DisposeRgn(clipRgn);
}
void VideoOutputQuartz::AspectChanged(float aspect)
{
VideoOutput::AspectChanged(aspect);
MoveResize();
// update transformation matrix with new aspect ratio
data->srcAspect = aspect;
Transform();
}
void VideoOutputQuartz::Zoom(int direction)
{
VideoOutput::Zoom(direction);
MoveResize();
}
void VideoOutputQuartz::InputChanged(int width, int height, float aspect)
{
VideoOutput::InputChanged(width, height, aspect);
DeleteQuartzBuffers();
CreateQuartzBuffers();
MoveResize();
scratchFrame = &(vbuffers[kNumBuffers]);
if (pauseFrame.buf)
delete [] pauseFrame.buf;
pauseFrame.height = scratchFrame->height;
pauseFrame.width = scratchFrame->width;
pauseFrame.bpp = scratchFrame->bpp;
pauseFrame.size = scratchFrame->size;
pauseFrame.buf = new unsigned char[pauseFrame.size];
// rebuild QuickTime decompressor
data->srcWidth = width;
data->srcHeight = height;
data->srcAspect = aspect;
BeginCodec(data->desiredXoff, data->desiredYoff,
data->desiredWidth, data->desiredHeight);
}
/* Return refresh rate of our display
*/
int VideoOutputQuartz::GetRefreshRate(void)
{
int refresh = 0;
CFDictionaryRef mode = CGDisplayCurrentMode(data->theDisplay);
if (mode)
{
CFNumberRef value = (CFNumberRef)
CFDictionaryGetValue(mode, kCGDisplayRefreshRate);
if (value)
{
CFNumberGetValue(value, kCFNumberIntType, &refresh);
}
}
return refresh;
}
bool VideoOutputQuartz::Init(int width, int height, float aspect,
WId winid, int winx, int winy,
int winw, int winh, WId embedid)
{
VERBOSE(VB_PLAYBACK, QString("VideoOutputQuartz::Init(width=%1, height=%2, aspect=%3, winid=%4\n winx=%5, winy=%6, winw=%7, winh=%8, WId embedid=%9)")
.arg(width)
.arg(height)
.arg(aspect)
.arg(winid)
.arg(winx)
.arg(winy)
.arg(winw)
.arg(winh)
.arg(embedid));
VideoOutput::InitBuffers(kNumBuffers, true, kNeedFreeFrames,
kPrebufferFrames, kKeepPrebuffer);
VideoOutput::Init(width, height, aspect, winid,
winx, winy, winw, winh, embedid);
if (!CreateQuartzBuffers())
return false;
scratchFrame = &(vbuffers[kNumBuffers]);
pauseFrame.height = scratchFrame->height;
pauseFrame.width = scratchFrame->width;
pauseFrame.bpp = scratchFrame->bpp;
pauseFrame.size = scratchFrame->size;
pauseFrame.buf = new unsigned char[pauseFrame.size];
data->srcWidth = width;
data->srcHeight = height;
data->srcAspect = aspect;
// Initialize QuickTime
if (EnterMovies())
{
puts("EnterMovies failed");
return false;
}
// Set up display, which also sets up codec
if (embedid)
{
embedding = true;
BeginDisplay(true, winx, winy, winw, winh);
}
else
{
BeginDisplay(data->drawInWindow, 0, 0, 0, 0);
}
MoveResize();
Started = true;
return true;
}
bool VideoOutputQuartz::BeginDisplay(bool windowed, int x, int y,
int w, int h)
{
data->theDisplay = CGMainDisplayID();
if (windowed)
{
// we reuse the GUI window
data->thePort = GetWindowPort(FrontNonFloatingWindow());
data->capturedDisplay = false;
}
else
{
// capture the main display
if (CGDisplayCapture(data->theDisplay))
{
puts("CGDisplayCapture failed");
return false;
}
if (data->changeResolution)
{
data->originalMode = CGDisplayCurrentMode(data->theDisplay);
data->newMode =
CGDisplayBestModeForParameters(data->theDisplay, 32,
data->srcWidth, data->srcHeight, NULL);
CGDisplaySwitchToMode(data->theDisplay, data->newMode);
}
CGDisplayHideCursor(data->theDisplay);
data->thePort = CreateNewPortForCGDisplayID((UInt32)data->theDisplay);
data->capturedDisplay = true;
}
if (!data->thePort)
{
puts("Failed to capture display port");
return false;
}
// set up everything else
return BeginCodec(x, y, w, h);
}
bool VideoOutputQuartz::BeginCodec(int x, int y, int w, int h)
{
int width, height;
width = data->srcWidth;
height = data->srcHeight;
// Set up decompressor to display YUV data
ImageDescriptionHandle yuvDesc =
(ImageDescriptionHandle) NewHandleClear(sizeof(ImageDescription));
HLock((Handle)yuvDesc);
(**yuvDesc).idSize = sizeof(ImageDescription);
(**yuvDesc).cType = kYUV420CodecType;
(**yuvDesc).version = 1;
(**yuvDesc).revisionLevel = 0;
(**yuvDesc).spatialQuality = codecLosslessQuality;
(**yuvDesc).width = width;
(**yuvDesc).height = height;
(**yuvDesc).hRes = Long2Fix(72);
(**yuvDesc).vRes = Long2Fix(72);
(**yuvDesc).depth = 24;
(**yuvDesc).frameCount = 0;
(**yuvDesc).dataSize = 0;
(**yuvDesc).clutID = -1;
HUnlock((Handle)yuvDesc);
if (DecompressSequenceBeginS(&data->seqID,
yuvDesc,
NULL,
0,
data->thePort,
NULL,
NULL,
NULL,
srcCopy,
NULL,
0, //codecFlagUseImageBuffer,
codecLosslessQuality,
bestSpeedCodec))
{
puts("DecompressSequenceBeginS failed");
return false;
}
SetDSequenceFlags(data->seqID,
codecDSequenceFlushInsteadOfDirtying,
codecDSequenceFlushInsteadOfDirtying);
// Set up storage area for one YUV frame (header + data)
data->pixelSize = (width * height * 3) / 2;
data->pixmapSize = sizeof(PlanarPixmapInfoYUV420) + data->pixelSize;
data->pixmap = (PlanarPixmapInfoYUV420 *) new char[data->pixmapSize];
long offset = sizeof(PlanarPixmapInfoYUV420);
data->pixelData = data->pixmap + offset;
// FIXME: This offset works for me, but I don't know why.
offset = 576;
data->pixmap->componentInfoY.offset = offset;
data->pixmap->componentInfoY.rowBytes = width;
offset += width * height;
data->pixmap->componentInfoCb.offset = offset;
data->pixmap->componentInfoCb.rowBytes = width / 2;
offset += (width * height) / 4;
data->pixmap->componentInfoCr.offset = offset;
data->pixmap->componentInfoCr.rowBytes = width / 2;
// Things won't work until the mask and transform are set properly
Mask(x, y, w, h);
Transform();
return true;
}
bool VideoOutputQuartz::CreateQuartzBuffers(void)
{
for (int i = 0; i < numbuffers + 1; i++)
{
vbuffers[i].height = XJ_height;
vbuffers[i].width = XJ_width;
vbuffers[i].bpp = 12;
vbuffers[i].size = XJ_height * XJ_width * 3 / 2;
vbuffers[i].codec = FMT_YV12;
vbuffers[i].buf = new unsigned char[vbuffers[i].size + 64];
memset(vbuffers[i].buf, 0, XJ_height * XJ_width);
memset(vbuffers[i].buf + XJ_height * XJ_width, 127,
XJ_height * XJ_width / 2);
}
return true;
}
void VideoOutputQuartz::EmbedInWidget(WId wid, int x, int y, int w, int h)
{
VERBOSE(VB_PLAYBACK, "Calling EmbedInWidget");
if (embedding)
return;
VideoOutput::EmbedInWidget(wid, x, y, w, h);
if (data->capturedDisplay)
{
// If we've been running full screen, we need to
// switch to the window port.
VERBOSE(VB_PLAYBACK, "Changing display for embedding");
data->capturedBeforeEmbed = true;
BeginDisplay(true, x, y, w, h);
}
else
{
// We're already on the window port, we just need
// to clip properly.
VERBOSE(VB_PLAYBACK, "Changing mask/transform");
data->capturedBeforeEmbed = false;
Mask(x, y, w, h);
Transform();
}
}
void VideoOutputQuartz::StopEmbedding(void)
{
if (!embedding)
return;
VideoOutput::StopEmbedding();
if (data->capturedBeforeEmbed)
{
// Recapture display
BeginDisplay(false, 0, 0, 0, 0);
}
else
{
// Reset clipping region
Mask(0, 0, 0, 0);
Transform();
}
}
void VideoOutputQuartz::PrepareFrame(VideoFrame *buffer, FrameScanType t)
{
(void)buffer;
(void)t;
}
void VideoOutputQuartz::Show(FrameScanType t)
{
(void)t;
// feed our buffered data to QuickTime
OSErr err;
err = DecompressSequenceFrameWhen(data->seqID,
(Ptr)data->pixmap,
data->pixmapSize,
0,
NULL,
NULL,
NULL);
if (err)
{
VERBOSE(VB_PLAYBACK, "DecompressSequenceFrameWhen failed");
}
}
void VideoOutputQuartz::DrawUnusedRects(void)
{
}
void VideoOutputQuartz::UpdatePauseFrame(void)
{
VideoFrame *pauseb = scratchFrame;
if (usedVideoBuffers.count() > 0)
pauseb = usedVideoBuffers.head();
memcpy(pauseFrame.buf, pauseb->buf, pauseb->size);
}
void VideoOutputQuartz::ProcessFrame(VideoFrame *frame, OSD *osd,
FilterChain *filterList,
NuppelVideoPlayer *pipPlayer)
{
if (!frame)
{
frame = scratchFrame;
CopyFrame(scratchFrame, &pauseFrame);
}
if (filterList)
filterList->ProcessFrame(frame);
ShowPip(frame, pipPlayer);
DisplayOSD(frame, osd);
// copy data to our buffer
memcpy(data->pixelData,
frame->buf, frame->size);
}
-------------- next part --------------
#ifndef VIDEOOUT_QUARTZ_H_
#define VIDEOOUT_QUARTZ_H_
struct QuartzData;
#include "videooutbase.h"
class VideoOutputQuartz : public VideoOutput
{
public:
VideoOutputQuartz();
~VideoOutputQuartz();
bool Init(int width, int height, float aspect, WId winid,
int winx, int winy, int winw, int winh, WId embedid = 0);
void PrepareFrame(VideoFrame *buffer, FrameScanType t);
void Show(FrameScanType);
void InputChanged(int width, int height, float aspect);
void AspectChanged(float aspect);
void Zoom(int direction);
void EmbedInWidget(WId wid, int x, int y, int w, int h);
void StopEmbedding(void);
int GetRefreshRate(void);
void DrawUnusedRects(void);
void UpdatePauseFrame(void);
void ProcessFrame(VideoFrame *frame, OSD *osd,
FilterChain *filterList,
NuppelVideoPlayer *pipPlayer);
private:
void Exit(void);
bool CreateQuartzBuffers(void);
void DeleteQuartzBuffers(void);
bool BeginDisplay(bool windowed, int x, int y, int w, int h);
void EndDisplay(void);
bool BeginCodec(int x, int y, int w, int h);
void EndCodec(void);
void Mask(int x, int y, int w, int h);
void Transform(void);
bool Started;
QuartzData * data;
VideoFrame * scratchFrame;
VideoFrame pauseFrame;
};
#endif
More information about the mythtv-dev
mailing list