man page MPSKernel section 3

MPSKernel(3)           MetalPerformanceShaders.framework          MPSKernel(3)

NAME

       MPSKernel

SYNOPSIS

       #import <MPSKernel.h>

       Inherits NSObject, <NSCopying>, and <NSSecureCoding>.

       Inherited by MPSBinaryImageKernel, MPSCNNBinaryKernel, MPSCNNKernel,
       MPSImageCopyToMatrix, MPSImageFindKeypoints, MPSImageHistogram,
       MPSMatrixBinaryKernel, MPSMatrixCopy, MPSMatrixMultiplication,
       MPSMatrixUnaryKernel, MPSNNGraph, MPSRNNMatrixInferenceLayer, and
       MPSUnaryImageKernel.

   Instance Methods
       (nonnull instancetype) - initWithDevice:
       (nonnull instancetype) - copyWithZone:device:
       (nullable instancetype) - initWithCoder:
       (nullable instancetype) - initWithCoder:device:
       () - MPSCopyAllocator

   Public Attributes
       const MTLRegion MPSRectNoClip

   Properties
       MPSKernelOptions options
       id< MTLDevice > device
       NSString * label

Detailed Description

MPSKernel.h MPSCore.framework

MPSKernel objects encode tuned image processing operations into a
MTLCommandBuffer.

This depends on Metal.framework The MPSKernel class is the base class
for all MPS objects. It defines a standard interface for MPS kernels.
You should not use the MPSKernel class directly. Instead, a number of
MPSKernel subclasses are available in MetalPerformanceShaders.framework
that define specific high-performance image processing operations.

The basic sequence for applying a MPSKernel to an image is as follows:

1. Create a MPSKernel corresponding to the operation you wish to
perform:

MPSImageSobel *sobel = [[MPSImageSobel alloc] initWithDevice: mtlDevice];

2. Encode the filter into a command buffer:

sobel.offset = ...;
sobel.clipRect = ...;
sobel.options = ...;
[sobel encodeToCommandBuffer: commandBuffer
sourceTexture: inputImage
destinationTexture: resultImage ];

Encoding the kernel merely encodes the operation into a
MTLCommandBuffer. It does not modify any pixels, yet. All MPSKernel
state has been copied to the command buffer. MPSKernels may be reused.
If the texture was previously operated on by another command encoder
(e.g. MTLRenderCommandEncoder), you should call -endEncoding on the
other encoder before encoding the filter.

Some MPS filters work in place (inputImage = resultImage) even in
situations where Metal might not normally allow in place operation on
textures. If in-place operation is desired, you may attempt to call
[MPSKernel encodeKernelInPlace...]. If the operation can not be
completed in place, then NO will be returned and you will have to
create a new result texture and try again. To make an in-place image
filter reliable, pass a fallback MPSCopyAllocator to the method to
create a new texture to write to in the event that a filter can not
operate in place.

(Repeat steps 2 for more filters, as desired.)

It should be self evident that step 2 may not be thread safe. That is, you can not have
multiple threads manipulating the same properties on the same MPSKernel object at the
same time and achieve coherent output. In common usage, the MPSKernel properties don't
often need to be changed from their default values, but if you need to apply the same
filter to multiple images on multiple threads with cropping / tiling, make additional
MPSKernel objects per thread. They are cheap. You can use multiple MPSKernel objects on
multiple threads, as long as only one thread is operating on any particular MPSKernel
object at a time.

3. After encoding any additional work to the command buffer using
other encoders, submit the MTLCommandBuffer to your
MTLCommandQueue, using:

[mtlCommandBuffer commit];

A MPSKernel can be saved to disk / network using NSCoders such as
NSKeyedArchiver. When decoding, the system default MTLDevice will be
chosen unless the NSCoder adopts the <MPSDeviceProvider> protocol. To
accomplish this, subclass or extend your unarchiver to add this method.

Method Documentation

- (nonnull instancetype) copyWithZone: (nullable NSZone *) zone(nullable
id< MTLDevice >) device
Make a copy of this MPSKernel for a new device -copyWithZone: will
call this API to make a copy of the MPSKernel on the same device. This
interface may also be called directly to make a copy of the MPSKernel
on a new device. Typically, the same MPSKernels should not be used to
encode kernels on multiple command buffers from multiple threads. Many
MPSKernels have mutable properties that might be changed by the other
thread while this one is trying to encode. If you need to use a
MPSKernel from multiple threads make a copy of it for each additional
thread using -copyWithZone: or -copyWithZone:device:

Parameters:
zone The NSZone in which to allocate the object
device The device for the new MPSKernel. If nil, then use
self.device.

Returns:
a pointer to a copy of this MPSKernel. This will fail, returning
nil if the device is not supported. Devices must be
MTLFeatureSet_iOS_GPUFamily2_v1 or later.

Reimplemented in MPSRNNMatrixInferenceLayer, and
MPSRNNImageInferenceLayer.

- (nullable instancetype) initWithCoder: (NSCoder *__nonnull) aDecoder
Called by NSCoder to decode MPSKernels This isn't the right interface
to decode a MPSKernel, but it is the one that NSCoder uses. To enable
your NSCoder (e.g. NSKeyedUnarchiver) to set which device to use extend
the object to adopt the MPSDeviceProvider protocol. Otherwise, the
Metal system default device will be used.

- (nullable instancetype) initWithCoder: (NSCoder *__nonnull)
aDecoder(nonnull id< MTLDevice >) device
NSSecureCoding compatability While the standard
NSSecureCoding/NSCoding method -initWithCoder: should work, since the
file can't know which device your data is allocated on, we have to
guess and may guess incorrectly. To avoid that problem, use
initWithCoder:device instead.

Parameters:
aDecoder The NSCoder subclass with your serialized MPSKernel
device The MTLDevice on which to make the MPSKernel

Returns:
A new MPSKernel object, or nil if failure.

Reimplemented in MPSCNNBinaryConvolution, MPSCNNBinaryFullyConnected,
MPSCNNConvolutionTranspose, MPSCNNConvolution, MPSCNNFullyConnected,
MPSRNNMatrixInferenceLayer, MPSRNNImageInferenceLayer, MPSCNNNeuron,
MPSImagePyramid, MPSCNNBinaryKernel, MPSBinaryImageKernel,
MPSCNNDilatedPoolingMax, MPSImageSobel, MPSCNNPoolingAverage,
MPSCNNPoolingL2Norm, MPSCNNCrossChannelNormalization, MPSCNNPooling,
MPSCNNPoolingMax, MPSImageHistogramSpecification,
MPSImageThresholdToZeroInverse, MPSCNNKernel, MPSImageThresholdToZero,
MPSCNNLocalContrastNormalization, MPSImageHistogramEqualization,
MPSUnaryImageKernel, MPSImageBox, MPSImageGaussianBlur, MPSMatrixCopy,
MPSImageStatisticsMean, MPSImageThresholdBinary,
MPSImageThresholdTruncate, MPSImageDilate, MPSImageScale,
MPSImageLanczosScale, MPSImageBilinearScale,
MPSImageStatisticsMeanAndVariance, MPSImageConvolution,
MPSImageThresholdBinaryInverse, MPSImageHistogram,
MPSCNNSpatialNormalization, MPSImageCopyToMatrix,
MPSImageFindKeypoints, MPSImageStatisticsMinAndMax, MPSImageMedian,
MPSImageAreaMax, and MPSNNGraph.

- (nonnull instancetype) initWithDevice: (nonnull id< MTLDevice >) device
Standard init with default properties per filter type

Parameters:
device The device that the filter will be used on. May not be NULL.

Returns:
a pointer to the newly initialized object. This will fail,
returning nil if the device is not supported. Devices must be
MTLFeatureSet_iOS_GPUFamily2_v1 or later.

Reimplemented in MPSCNNBinaryConvolution, MPSCNNBinaryFullyConnected,
MPSCNNKernel, MPSCNNConvolutionTranspose, MPSCNNConvolution,
MPSCNNFullyConnected, MPSRNNMatrixInferenceLayer,
MPSRNNImageInferenceLayer, MPSBinaryImageKernel, MPSImagePyramid,
MPSCNNNeuronReLUN, MPSCNNNeuronELU, MPSImageSobel,
MPSCNNCrossChannelNormalization, MPSCNNPooling, MPSCNNNeuronSoftPlus,
MPSCNNNeuronSoftSign, MPSImageThresholdToZeroInverse, MPSCNNNeuronTanH,
MPSCNNNeuronAbsolute, MPSCNNBinaryKernel, MPSImageThresholdToZero,
MPSCNNNeuronHardSigmoid, MPSCNNLocalContrastNormalization,
MPSUnaryImageKernel, MPSImageBox, MPSImageGaussianBlur,
MPSImageStatisticsMean, MPSImageThresholdBinary,
MPSImageThresholdTruncate, MPSImageDilate, MPSImageScale,
MPSCNNNeuronReLU, MPSCNNNeuronPReLU, MPSCNNNeuronSigmoid,
MPSImageLanczosScale, MPSImageBilinearScale,
MPSImageStatisticsMeanAndVariance, MPSMatrixMultiplication,
MPSMatrixVectorMultiplication, MPSImageThresholdBinaryInverse,
MPSImageArithmetic, MPSImageAdd, MPSImageSubtract, MPSImageMultiply,
MPSImageDivide, MPSCNNNeuronLinear, MPSCNNSpatialNormalization,
MPSImageFindKeypoints, MPSCNNUpsampling, MPSImageStatisticsMinAndMax,
MPSImageMedian, MPSImageAreaMax, and MPSMatrixCopy.

- MPSCopyAllocator
MPSImageKernel.h MetalPerformanceShaders.framework

A block to make a copy of sourceTexture for MPSKernels that can only
execute out of place. Some MPSKernel objects may not be able to
operate in place. When that occurs, and in-place operation is
requested, MPS will call back to this block to get a new texture to
return instead. To avoid spending long periods of time allocating pages
to back the MTLTexture, the block should attempt to reuse textures. The
texture returned from the MPSCopyAllocator will be returned instead of
the sourceTexture from the MPSKernel method on return.

// A MPSCopyAllocator to handle cases where in-place operation fails.
MPSCopyAllocator myAllocator = ^id <MTLTexture>( MPSKernel * __nonnull filter,
__nonnull id <MTLCommandBuffer> cmdBuf,
__nonnull id <MTLTexture> sourceTexture)
{
MTLPixelFormat format = sourceTexture.pixelFormat; // FIXME: is this format writable?
MTLTextureDescriptor *d = [MTLTextureDescriptor texture2DDescriptorWithPixelFormat: format
width: sourceTexture.width
height: sourceTexture.height
mipmapped: NO];
d.usage = MTLTextureUsageShaderRead | MTLTextureUsageShaderWrite;

//FIXME: Allocating a new texture each time is slow. They take up to 1 ms each.
// There are not too many milliseconds in a video frame! You can recycle
// old textures (or MTLBuffers and make textures from them) and reuse
// the memory here.
id <MTLTexture> result = [cmdBuf.device newTextureWithDescriptor: d];

// FIXME: If there is any metadata associated with sourceTexture such as colorspace
// information, MTLResource.label, MTLResource.cpuCacheMode mode,
// MTLResource.MTLPurgeableState, etc., it may need to be similarly associated
// with the new texture to avoid losing your metadata.

// FIXME: If filter.clipRect doesn't cover the entire image, you may need to copy
// pixels from sourceTexture to the new texture or regions of the new texture
// will be uninitialized. You can make a MTLCommandEncoder to encode work on
// the MTLCommandBuffer here to do that work, if necessary. It will be
// scheduled to run immediately before the MPSKernel work. Do not call
// [MTLCommandBuffer enqueue/commit/waitUntilCompleted/waitUntilScheduled]
// in the MPSCopyAllocator block. Make sure to call -endEncoding on the
// MTLCommandEncoder so that the MTLCommandBuffer has no active encoder
// before returning.

// CAUTION: The next command placed on the MTLCommandBuffer after the MPSCopyAllocator
// returns is almost assuredly going to be encoded with a MTLComputeCommandEncoder.
// Creating any other type of encoder in the MPSCopyAllocator will probably cost
// an additional 0.5 ms of both CPU _AND_ GPU time (or more!) due to a double
// mode switch penalty.

// CAUTION: If other objects (in addition to the caller of -encodeToCommandBuffer:inPlaceTexture:...)
// own a reference to sourceTexture, they may need to be notified that
// sourceTexture has been replaced so that they can release that resource
// and adopt the new texture.

// The reference to sourceTexture owned by the caller of
// -encodeToCommandBuffer:inPlaceTexture... will be released by
// -encodeToCommandBuffer:inPlaceTexture:... after the kernel is encoded if
// and only if the MPSCopyAllocator is called, and the operation is successfully
// encoded out of place.

return result;
// d is autoreleased
};

If nil is returned by the allocator, NO will be returned by the
calling function.

When the MPSCopyAllocator is called, no MTLCommandEncoder is active on
the commandBuffer. You may create a MTLCommandEncoder in the block to
initialize the texture. Make sure to call -endEncoding on it before
returning, if you do.

Parameters:
filter A valid pointer to the MPSKernel that is calling the
MPSCopyAllocator. From it you can get the clipRect of the intended
operation.
commandBuffer A valid MTLCommandBuffer. It can be used to obtain
the device against which to allocate the new texture. You may also
enqueue operations on the commandBuffer to initialize the texture
on a encoder allocated in the block. You may not submit, enqueue or
wait for scheduling/completion of the command buffer.
sourceTexture The texture that is providing the source image for
the filter. You may wish to use its size and MTLPixelFormat for the
new texture, but it is not requred.

Returns:
A new valid MTLTexture to use as the destination for the MPSKernel.
If the calling function succeeds, its texture parameter will be
overwritten with a pointer to this texture. If the calling function
fails (highly unlikely, except for user error) then the texture
will be released before the calling function returns.

Member Data Documentation

   - (const MTLRegion) MPSRectNoClip
       MPSRectNoClip  This is a special constant to indicate no clipping is to
       be done. The entire image will be used. This is the default clipping
       rectangle or the input extent for MPSKernels.

Property Documentation

   - device [read],  [nonatomic],  [retain]
       The device on which the kernel will be used

   - label [read],  [write],  [atomic],  [copy]
       A string to help identify this object.

   - options [read],  [write],  [nonatomic],  [assign]
       The set of options used to run the kernel. MPSKernelOptions

Author

       Generated automatically by Doxygen for
       MetalPerformanceShaders.framework from the source code.





Version MetalPerformanceShaders-Thu2Jul 13 2017                   MPSKernel(3)


Mac OS X 10.13.1 - Generated Tue Nov  7 18:17:51 CST 2017